On our final day, we will take a look at some of the conundrums and controversies related to authentic assessment. These can be roughly grouped into three topics – questions of validity and reliability, questions of barriers to adopting authentic assessment in Higher Education, and wider global political and philosophical issues relating to the purpose of Higher Education and universities as institutions.
Choose a controversy!
These topics are very complex and we realise the 15 minutes may be a bit of an ask for this page! You are welcome to skim through, concentrate on the controversy topic that interests you most, and to explore the references and links later in your own time. In addition, rather than trying to address all discussion questions, choose the one you are most interested in, to comment on. Of course if you are keen and have the time, feel free to explore all three!
Controversy 1: Questions of validity and reliability
Validity and reliability have long been mainstays of “good assessment” at all levels of education, based on a positivist, statistical approach that values standardisation. This has resulted in a tendency to use objective testing such as multiple choice, or supervised examinations with formulaic tasks that are marked against rubrics by people who do not know the students. However this type of rigorous control results in assessments that lack authenticity and therefore validity of a different nature. The idiosyncrasies of individual response to a task and the faithfulness of an assessment to real life conditions are now also valued. (See Hathcoat et al, 2016, and Wiggins, 1993).
The newly evolved push for authentic assessments requires criterion-referenced grading rather than norm-referenced grading, since the assessment has to be true to a complex, work-like task, not to an abstract statistical norm. The continued practice in some universities of using norm-referenced methods of assessment and grading for administration and funding purposes are in opposition to the trend towards making assessments more authentic.
The concern that authentic assessments are not being held to the “same high standards of reliability and validity” as traditional assessments (Hathcoat et al, 2016 p. 16) can be countered with the idea that without authenticity, the validity of an assessment can be challenged also. Other standards are considered important to authentic assessment, such as transparency, effective rubrics and close alignment with course outcomes and professional standards. In addition, authentic assessments require many instances of assessment in a holistic process over time, and is not suited to one high stakes assessment event.
What are your thoughts on the issues of “validity and reliability” in authentic assessment? Is it realistic to expect statistically derived concepts to be applied to assessments comprising messy, ill-defined tasks that mirror the professional or work environment? What about areas such as medicine, where standard procedures are used routinely and must be learned thoroughly or lives will be at risk? How should competency with such procedures be tested? Does the need for authentic assessment prevent the use of objective, standardised tests for certain purposes?
Controversy 2: Systemic barriers to the adoption of authentic assessment
The move to authentic assessment in the Higher Education sector is part of a rapid succession of changes to traditional university education due to technological developments and globalisation. Not all universities are ready for this change.
One of the systemic issues identified by Boud and Falchikov (2005), is the dominance of summative assessment concerns within universities. The quality and consistency of assessments for the sifting out of successful and unsuccessful graduates is a preoccupation of university administrations and external bodies such as professional associations. For them, processes such as this comprise signifiers of accountability and transparency as well as of quality. Related to this issue, Boud and Falchikov also identified a persistent use of traditional norm-referenced assessment grading, a system that makes authentic assessment difficult to grade, instead of criterion-referenced grading.
Other barriers are around the issue that university staffing and resources may not always be sufficient to meet the demands of providing authentic assessment environments, contexts and processes. (See Jackson et al reference below for a discussion about difficulties in obtaining Work Integrated Learning placements, for example). It seems that it is often the case that neither professions/industry, nor universities as institutions, are ready for well-designed authentic assessment activities that have students engaged with the professional communities for whom they are being trained.
Have you encountered any of these barriers? Have you been able to find work-arounds? Bearing in mind that some theorists insist that authentic assessment can approximate, rather than exactly replicate, a real life task, can you think of ways to assess authentically in spite of these systemic barriers?
Controversy 3: What is the purpose of Higher Education and the institution of the university, anyway?
Authentic assessment as an approach has been explored and adopted since the 1990s partly as a result of the growing concern about the employability of graduates. University courses have become more vocationally oriented and also commercialised, or some might say “commodified.” The view of the student as customer purchasing their learning and qualifications has resulted in student expectations that the outcome of completing a course is that of being employable, as the main return on investment. The controversy around this is summed up by Kinash et al (2018):
“While graduate employability is the dominant discourse, the literature is replete with debate about whether the purpose/s of higher education are vocational training, career development, economic growth, global competitiveness and/or personal development, knowledge capital, research and critical perspectives.” (Kinash et al, 2018, p. )
Universities are now operating in a market economy, competing with each other, and often ranked according to employment outcomes. Those who do not agree with this agenda value a Higher Education that is aiming at personal development, knowledge building, critical perspectives and research achievements. A social critique is that market based models of Higher Education leave people out, with a university education available to the few who can afford fees and internships, and the individual blamed for failure when it could be a market failure (Kinash et al, 2018, p. ). However those who favour a more vocational approach to Higher Education would argue the opposite, that focusing on narrow academic and research endeavours will result in an elitist system that deprives the general population of a higher education. No doubt the answer is somewhere in between!
Where do we strike a balance between academic scholarship and research on the one hand, and on the other hand, employability and readiness for work, in assessment and learning within higher education? Does one necessarily count out the other or do they go together? Share your thoughts on this question or anything else you would like to discuss.
Boud, David & Falchikov, Nancy (2005), “Redesigning assessment for learning beyond higher education” in Higher Education in a Changing World, Herdsa 2005 Conference Proceedings. http://conference.herdsa.org.au/2005/pdf/refereed/paper_398.pdf
Groves, Nancy, (2012) “Authentic Assessment – what does it mean for staff, students and sector?” In Guardian (education feature) 2012 https://www.theguardian.com/higher-education-network/blog/2012/oct/24/authentic-assessment-university-teaching-learning
Hathcoat, J. D., Penn, J. D., Barnes, L. L., B., & Comer, J. C. (2016). “A second dystopia in education: Validity issues in authentic assessment practices.” Research in Higher Education, 57(7), 892-912. doi: http://dx.doi.org.virtual.anu.edu.au/10.1007/s11162-016-9407-1
Jackson, Denise; Rowbottom, David; Ferns, Sonia & McLaren, Diane (2017) Employer understanding of Work-Integrated Learning and the challenges of engaging in work placement opportunities, Studies in Continuing Education, 39:1, 35-51, DOI:10.1080/0158037X.2016.1228624
Kinesh, Shelley, McGillivray, Laura, Crane, Linda (2087) ” Do University students, alumni, educators and employers link assessment and graduate employability?” in Higher Education Research and Development, Vol 37, No. 2, pp. 301-315 https://www.tandfonline.com/doi/full/10.1080/07294360.2017.1370439
McAlister, B. K., Custer R. L., Schell, J., Scott, J. S., & Hoepfl, M. (2000) Using Authentic Assessment in Vocational Education: ERIC Monograph (Information Series No. 381). https://www.researchgate.net/publication/234753168_Using_Authentic_Assessment_in_Vocational_Education_Information_Series_No_381
McDermott, Roger, Zarb, Mark, Daniels, Mats, Nylen, Aletta, Pears, Arnold; Isomöttönen, (2017): “The authenticity of “authentic” assessment some faculty perceptions” in 2017 IEEE Frontiers in Education Conference Proceedings Indianapolis, USA https://ieeexplore.ieee.org/document/8190604