Assessment and Feedback

Day 4: Designing authentic assessment

Image of tiles with letters for the word "assess"

How does authentic assessment relate to the style of teaching and learning?

Authentic assessment requires authentic learning experiences, as expressed so succinctly in the abstract of the article by Robyn Collins (2013), which we shared in our post on Day 2. This approach is based on the concept of “constructivist learning” theory (see our previous Coffee course on Learning Theory).  Related concepts are: active learning, situated learning and work-based learning.  Where there is no realistic access to an actual work place, use is made of simulations, role plays and case studies.  

question mark

Discussion

Do you think authentic assessment is more relevant to formative assessment, summative assessment, or both?  What role do you think formative assessment would play in an authentic learning and assessment program?  Is there a place for tests and essays in such a program?

Authentic assessment design

We will refer to two useful articles here that provide frameworks for authentic assessment design.  

Using a framework to review assessments for authenticity

The literature survey by Ashford Rowe et al (2014) aimed to isolate critical elements for authentic assessment, then developed a framework of 8 principles, to guide the development of authentic assessment (as discussed yesterday).   The researchers then applied the framework to review and redevelop an assessment, to test it out with students and then evaluate against the framework.  

They then created a table using  8 questions based on the principles to set out how and why the existing assessment needed redevelopment in order to meet the authenticity requirements. The table makes for interesting reading, as does their second, evaluative table after the assessment had been conducted and students surveyed on their experience (please see reference and link under References below).

Here is an excerpt (for the first critical element) from their first table in which they examined their existing assessments: 

Critical question
Assessment before
Proposal for assessment after redesign
Designer rationale
To what extent does the assessment activity challenge the assessed student? Requires the student to answer non-applied theory questions as a test of  memory Will require students to apply the theoretical content in practice The intention was to increase the degree of challenge on the student by expecting them to apply what they had learnt in theory to achieve an applicable outcome.  

ActivityTools, symbol for work

Choose one of your assessments that is more traditional and then, select one or two of the 8 critical elements and use it to create a question/s and a proposal for a more authentic assessment, that addresses the question/s, like the example shown above. The example above provides a very general statement as to the proposed change, and you are encouraged to say something a little more specific about your assessment if you wish, when sharing.  For example, you might say something like “will require the student to come up with a project plan”.

To help you with this activity, you might like to first re-think your assessment and the relevant teaching and learning activities, using the following questions:

  • What is your course/teaching activity?
  • What is the authentic context that your students would be working in?
  • What sorts of activities or skills are needed in that authentic context?  
  • Do the learning outcomes reflect the authentic context – if not, try re-writing the relevant learning outcome/s
  • What assessment would be useful to see if students have those skills? 

Share any insights you gained from trying this activity, with others here.

Ensuring graduate outcomes and the professional context are reflected in the assessment

In another article,  Villaroel et al (2018) identified through a literature review,  three dimensions representing “the essence” of authentic assessment:  (1) Realism, (2) Cognitive Challenge and (4) Evaluative judgment.

The authors develop a four-step blueprint, using these dimensions, for devising authentic assessments that translate them into the Higher Education sector.   This involves the teacher in considering the graduate outcome requirements for their program, their knowledge of work requirements in the field, creating a meaningful task based on this, co-creating assessment criteria with students (or at least involving them in discussions about the criteria), and providing both formative and summative feedback that is useful and sustainable to students.

Go here for a text alternative to this diagram

Diagram of model to build authentic assessment

 

question markDiscussion

Take a look at the graduate outcomes for the program in which you teach, and also consider the type of work context for which you might be preparing the program’s students.  Does this help to inform your thinking about creating authentic assessments at the course level?

Villaroel et al, (2018), p. 847 

Dilemmas presented

In considering implementing a more authentic approach to your assessment, you are possibly already presented with issues and difficulties.  For example, how easy is it to obtain this professional or industry exposure for undergraduate students?  What private companies would be willing to collaborate with a university course and its students in this way? We are going to be examining more closely issues and dilemmas that authentic assessment may imply, in some contexts, in the coming days.

question mark

Discussion

Looking at the above example, and thinking about your own context, what problems and issues would you foresee in designing authentic assessments that reflect the graduate workplace?  

References

Ashford-Rowe, Kevin,  Herrington, Janice & Brown,  Christine  (2014) “Establishing the critical elements that determine authentic assessment,” Assessment & Evaluation in Higher Education, 39:2, 205-222, https://doi.org/10.1080/02602938.2013.819566 

Collins, Robyn, 2013, “Authentic assessment:  assessment for learning” in Curriculum and Leadership Journal, Vol. 11, Issue 7, May 2013.  http://www.curriculum.edu.au/leader/authentic_assessment_assessment_for_learning,36251.html?issueID=12745 

 Villarroel, Veronica,  Bloxham, Susan, Bruna, Danila, Bruna, Carola,  & Herrera-Seda, Constanza  (2018) “Authentic assessment: creating a blueprint for course design,”  Assessment & Evaluation in Higher Education, 43:5, 840-854, DOI: 10.1080/02602938.2017.1412396   https://doi.org/10.1080/02602938.2017.1412396 

20 thoughts on “Day 4: Designing authentic assessment

  1. In regard to the first question of whether authentic assessments are better for formative and summative, I think we need to think about why we are assessing. If we are assessing whether somebody is ready to do something in the real world (which would be Competency Based Training and the way the Vocational Educational sector works) then it should be summative. Thats not to say it shouldn’t be formative. Indeed a formative authentic assessment coupled with a summative authetic assessment maybe idea. Howe ver a authentic formative assessment and a summative “traditional” assessment may not be a good coupling

    1. Hi David, thanks for your thoughts. I agree with what you are saying – possibly formative assessment would provide the “safe environment to fail” we were discussing in previous days, with an authentic summative assessment that can be done when the student feels ready. Yes, this does approach the more VET-oriented competency based assessment – but I am not convinced VET ever got that right either. No one ever worked out a way to distinguish outstanding achievement from “just doing the task.” Nor did anyone ever successfully deal with the issue of “knowledges” that are more abstract than can be assessed with a set of practical tasks. These are my own personal thoughts from my experience in VET then in Higher Education. I would be interested in what others think.

    2. David, in every course I am assessing if the student is competent to do something in the real world. But along the way, I give little tests to help them get there. These little tests are part of the final result, but small enough so that, as Jill puts it, a “safe environment to fail”.

      I don’t see a contradiction between VET competency based assessment, and university assessment. University assessment goes on to measure when a student is more than competent. As mentioned previously, my rule of thumb is that a “Credit” is “Competent”.

  2. Authentic assessment must be summative, like all assessment, in my view. Small summative assessment tasks can be used to help measure, and influence the student’s learning. But because these are small, the will be less authentic. As an example, I quiz students on terms they need to know, equations, and other basic knowledge. This is assessed to encourage them to do the exercise, and counts towards their final grade (otherwise they would not do it, and it would not be “assessment”). This gets them ready for the more complex exercises, and also so I can see who needs help.

    Ashford Rowe et al’s (2014) framework doesn’t look much use to me, as I see a role for small simple inauthentic assessments for basic skills, alongside complex “authentic” ones. The simple questions are not intended to challenge the student. It would be pointless to start with challenging questions, before the student has learned the basics. This would just demoralize them, and place them under severe emotional stress.

    As a student myself, I have had the experience of being presented with learning material which makes no sense. This happened today in a class where we were presented with a complex diagram. Four out of forty of us could not make any sense of the diagram, and were left floundering.
    In contrast, our coffee course notes have alternative text for the diagrams, which I find much easier to understand, and can work my way through. After I have understood the text description, I can have a go at interpreting the diagram. As I teach students with English as a second language, and a wide range of technical skills, I try to make sure I explain any terms used, and offer alternative step by step learning for those having difficulty.

    But while it is clearly laid out, I still could not understand the point Villaroel et al (2018) were making about connecting about graduate outcomes, work context, and authentic assessment in courses. To me learning design is a top down process, starting with external industry defined skills requirements, translating these into program, and then course requirements, then assessment and lastly learning resources to support the assessment. Is this was what they were telling us to do? What would be the alternative? If you started by deciding to teach something, how would you know it is something the student needs to learn? How would you know if you could test it? If you can’t test it, or they don’t need to know it, why teach it?

    1. Hi Tom, thanks for both your comments above. I like your practice of providing small tests that build up confidence, and take your point that it is difficult to convince students to do an activity unless it is an assessment. I think whether you call these small tests formative or summative almost becomes a question of semantics when discussed like this – however I see your small tests as formative assessments. Although they only contribute a very small percentage to the final assessment mark, they are helping the student build their skills and confidence, hence “formative.”
      Regarding your question about what is meant by connecting graduate outcomes with course outcomes and assessment design, I think you have answered your own question – by designing assessments with top level outcomes in mind, you are meeting the requirements of constructive alignment – something that students value. Thanks for raising these questions and giving me the opportunity to interrogate these concepts further!

  3. I agree that the main issue we encounter with undergraduate students is that they need to have a foundation of knowledge – formulas, theories, laws, etc. – and we need to assess their competency with these skills before we can go on using them in a more complex authentic environment. How can we expect the students to apply their knowledge to the real world when they barely understand the concepts by themselves? It requires a lot more effort to think creatively, yet logically and feel confident enough to apply these skills. So I really think that authentic assessment is more of a longer-term plan. For example, maybe in first-year subjects, it is more valuable to have traditional assessment tasks such as formative quizzes and homework assignments and the summative end of subject exam to ensure that the students feel comfortable in their grasp of the theories. Then, after the students have a solid base to work with, we can begin to introduce the applicability of these skills. Unfortunately, with subjects such as physics, students are still learning the foundations of a topic in the second year, which really only leaves the third year specialist subjects for more authentic tasks. This brings back another issue of whether it’s fair to introduce a whole new assessment method on the students’ final year, which could affect their overall averages and placement in postgrad positions.

    So basically, I guess I’m left trying to find a way to include small authentic assessment tasks in each subject, on top of all the tests and assignments the students are already doing (not the most time-conscientious); or maybe having a separate subject solely dedicated to authentic learning, which is advertised as focusing on preparing students for real-world applications (with the risk that there will be students who won’t take the course and will be left bridging the gap between uni and vocation by themselves)?

    1. Sarah, thanks so much for this insight! I wanted to highlight your comment about authentic assessment being a larger, big picture sort of goal. Approaching it at a curriculum level is a really great strategy, which might help to mitigate some of the concerns that were discussed in the previous post around students struggling with too much self-directed learning. Scaffolding into this from first year, with more traditional assignments with some “authentic” elements, to second- and third-year courses having increasingly more authentic, self-directed tasks, sounds like a great design!

  4. Authenticity in marketing assessment is straightforward. Take a product to market or take a market to product. Find a problem experienced by an audience, develop a solution to solve a part or all of the problem, and then monetize.

    The challenge is whether our students have
    a) the necessary pre-existing skills to make an authentic solution to the authentic assessment
    b) the understanding of the domain to recognise the task is authentic, purposeful, and a good use of their efforts
    c) the access to the requisite resources within the context of a teaching environment (eg 10 hours per week), and the task is scaled to fit.

    On the production side, do we have
    a) support from our Dean, Director, Head of School to expend energy on education outcomes, when that education could be assessed with an automatically marked online quiz (30%, provided by publisher, multiple options to resit), a single exam (40%, 1 essay question), and group assignment with a mandatory group of 6 (30%, 20% paper, 10% ‘individual’ contribution peer mark) to minimize marking time and expense

    b) access to the authentic task in an ethical and non exploitive or non-legal liability creating environment. An authentic task in marketing is selling a product to a consumer – should we send our students to sell things to customers? What liability to we incur if the customer has a problem? What liability will a risk adverse Dean see in the project that has real world success/failure events?

    c) ethical clearance to engage in authentic tasks around projects for which we have ownership (and therefore control of task, but also direct benefit flow from student efforts).

    I mean, this upcoming semester, I have a two Services Marketing courses, I run a services marketing operation (lego serious play) within the RSM, I could set the students to work on aspects of the LSP service project that would result in a direct benefit to my school and to me, from the compulsory labour of my students in a nonvoluntary assessment task.

    I want authentic tasks to work, I just think we’re in a 0.9a Beta Build of the assessment process here, and our bosses are looking surprised it’s not the polished Version 10 Advanced Edition, because Education, not that hard right? Just y’know, teach stuff, and put your real effort into research, yeah?

    1. Hi Stephen, wow you have so clearly set out all of the typical barriers experienced in attempting to make assessment true to the professional work place – and they are all big picture barriers which are beyond the control of individual lecturers. It seems that the closer to authenticity we get, the more risks we face. One thought I had regarding the example you gave of marketing your Lego Serious play marketing operation, is that you do make sure the students give informed consent – which I guess alludes to your point about ethical clearance. Your point about ready-made, automated online assessments is also a very pertinent one as of course it is less resource and time intensive. I agree with you that we are at “0.9a Beta build stage” in terms of workable authentic assessments, and the pressure to put all the effort and time into research as a priority over teaching.

  5. Similar to David, I believe authentic assessment should encompass both formative and summative assessment, depending on the reason for the assessment. Formative authentic assessment helps to scaffold skills towards a larger summative assessment. Just like content, authentic learning does not happen overnight. Students need opportunities to practice and learn. For example, while students may be summatively assessed on a specific role-play negotiation towards the end of the semester, the series of formative negotiations leading up to it provide opportunities for reflection and improvement – a very authentic process in the workplace.

    Whether there is a place for tests and essays really depends on the type of workplace you are preparing your students for. Essays are important for those who are inclined towards research. However, there are alternative written assessment formats, similar to, yet slightly different from essays, that may be more authentic for other careers. For instance, legal research memoranda and client letters for law, policy briefs and reports for many policy-oriented career pathways, writing a business case or project proposal in engineering and economics/business. The potential for essay-esq authentic assessment is only limited by our imaginations.

    1. Hi Bhavani, I love the idea of “essay-esque” assessments! I did a social science degree myself and wrote a million essays throughout my undergraduate, and would have loved a bit more variety in the kinds of writing I needed to do. I recall struggling a lot in a upper-year course when asked to write in different modes because I had been so enculturated into the essay format! Perhaps as educators we need to stress the transferable skills that are gained through degrees – this is something I think particularly needs to be done for HDR students who are not going into academia! Writing a thesis is certainly only an “authentic” assessment if you are planning to become a researcher, but data analysis, literature reviewing, and writing are all transferable skills.

      1. Hi Katie,

        Couldn’t agree more about stressing transferable skills, across all levels! I have a lot of undergrads and coursework postgrads who don’t realise all of the transferable skills that we are trying to develop within them, let alone why.

  6. Im not sure if I have a straight answer for if authentic assessment is relevant for formative or summative assessment, but I guess overall I think what we should be doing mostly is focusing on the formative – in so much as we should be facilitating students to better learning outcomes. And failure is one of the greatest ways to learn, so we need to provide environments for students to fail and that is why I think tasks that don’t involve summative assessment are so important. In the Vietnam Field School students essentially complete the same research task twice. Given that research suggests that formative assessment is critical because as soon as students see a number they don’t read the comments, I deliberately make the first research task compulsory but unmarked (they get detailed feedback though). That means that students focus on mastering the task and stretching beyond their capabilities knowing that they risk failing but that the only consequences of this are deep learning of what to avoid next time, and reflection on how to improve. However if we are going to use assessment as a way to encourage students to stretch themselves we need to support students too. In VFS my approach is fundamentally based on independent learning and throwing students in the deep end, this is incredible effective but I also spend a huge amount of the course offering the emotional support that students need to feel comfortable to take risks before they feel ready and to pick themselves up and learn from failures and do it all again. This occurs in group debrief sessions, but also in one on one chats.

    1. I wanted to pick up on what Edwina wrote about VFS-students completing the same research task twice, because it is something I also applied in my teaching practice (English as a Second Language-Classes in adult education). Like Edwina, I provided qualitative and detailed feedback on the first draft of writing assignments, but I did not assign a mark at that point. I then had students brainstorm on how they could improve their task. A couple of weeks later, they had to resubmit. Some students would completely redraft their text, while others (usually a minority) only made superficial improvements (Because they did not want to put in more time? Because they were confident that they would score high enough? Hard to say!) Like Edwina indicated, this method forced students to read my feedback, and it also mitigated their stress levels (since their first draft would be unmarked). I also feel that it was really beneficial to their creativity. And the in-class brainstorm added an element of collaborative learning, which many students really enjoyed! (Those that did not feel comfortable sharing their work did not have to, by the way.)

      And here are my two cents on formative vs summative assessment, I try to make both as authentic as I can. Formative assessments are shorter in nature, so naturally the context will be a little less rich and the assessment rubric will be simpler. Summative assessments, on the other hand can be multi-staged and more developed.

  7. When I was studying to teach English to speakers of other languages, we were all encouraged to observe classes (which is always great PD) but I was the only one allowed to do a teaching practicum for the course because I was already a qualified primary school teacher. My fellow classmates were only allowed to do some one-on-one tutoring and a one-off 10-minute lesson to our class. It astonished me that they could become qualified teachers having never had any real classroom experience nor having received detailed feedback on how they taught a series of lessons! I’ve met a number of people since who say they studied ESL teaching but never taught because they didn’t have the confidence to do so and I can see why! Maybe these people would have been wonderful teachers but their training did not give them the opportunity to prove to themselves what they could do!

    I also totally agree with Bhavani that students should be given the opportunity to perfect the art of all sorts of writing! Students who have developed writing skills beyond the essay will feel much more equipped to deal with life after uni.

    1. Thanks Rowena and Bhavani for your thoughts,

      I agree that students in HE definitely should practice their writing (and thinking!) skills, and one of the purposes of HE is surely to gain the skills to express analyse, and critique ideas, and one’s ability for this can be tested in a range of writing as well as speaking tasks. The 3000 word essay is an easy fall-back for teachers but I do think it is worth thinking about other writing tasks that reflect the world we live in, as suggested by Bhavani and Katie.

      Rowena, it is curious that people learning TESOL skills are not required to do a proper teaching practicum within the context they intend to work (ie, adult education). Such a practicum, if supported properly would be invaluable. I am one of those who did have to do a practicum for a Masters in TESOL that I completed and it was quite valuable and also an eye-opener for me on the level of skills required to facilitate a class room full of people who cannot communicate with me because none of them speak a word of English! Mime and performance were definitely part of the mix!

  8. I think authentic assessment is important in both formative and summative assessment. Like previous commenters and my previous comments have said, students may not yet have the skills, knowledge, or understanding of the domain to feel successful in completing an authentic assessment task. I think that formative assessment can be run in a more traditional sense of testing one skill or area of knowledge, but these can then build to a summative authentic task. In medicine we talk about “spiral learning” in that we approach the same topic many times, each time layering on more complexity and knowledge rather than assuming we will learn (and remember!) everything about a particular topic in one go.


    From personal experience, whilst I think it’s okay to have formative assessment more on the “traditional” side and building towards authentic assessment, I also think that it is important to give student adequate exposure to any new or atypical assessment formats. For example, in medicine we use written exams where you are unable to turn the pages of your exam book forwards or backwards (as you gradually get more information about the case in order to try to mimic real life). Although the idea sounds easy enough to explain, it’s helpful to have a practice examination, or give out practice papers so that students aren’t overly stressed about the method of examination when the time comes.

  9. I think authentic assessment should be both formative and summative, but I reckon in some disciplines it may be complex to find a fair balance between both.
    as commented in some of the previous posts, in the case of undergraduates, it may pretty complicated to design an industry/work-based assessment, especially in first year or introductory courses. As other have commented, students may feel overwhelm and not confident enough with the subject to complete a reflective and real life related tasks. I think one of the big issues in designing authenthic assessment for undergraduates would be to find the correct balance between what they know, what they will learn or have learnt in the course and their understanding of the application of this knowledge to their discipline

    1. Hi Sofia, thanks for this comment. I have found when working with academics trying to include more authentic assessment that it can be a challenge when students have just come straight from school and may not have the relevant experience to begin applying their knowledge right away. As we know from Bloom’s taxonomy, applying knowledge requires the foundations of remembering and understanding (https://tips.uark.edu/using-blooms-taxonomy/). But I have found highlighting WHY something is being done (i.e., as it is how things are done in an authentic work environment) can help bridge the gap and help students make sense of why they are learning specific things.

  10. On the question of formative or summative, I always try to go with both. A formative assessment not only will give the student an opportunity to test their knowledge but also can introduce them to the format of the summative.
    On the question what is authentic in HE, I found this a bit more challenging in my undergraduate teaching (in medicine, the targets are clear, the graduate outcomes are given, and we know what the work environment will look like). In my undegraduate science courses on the other hand I have a variety of students, some of whom don’t yet know what they want to do in the future, others do know, but that creates a mixed group, between academia, public service, industry or medicine. So, transferable skills are the most useful for them all. In the past I ran a neuroscience course, where the course assignment was a group task to investigate a particular neurological disease, and prepare a group presentation where they explain the science, the clinical picture and the present therapeutic approaches of this particular disease. As the groups divided the tasks, everyone could pick up on their interest (clinical, basic science, pharmaceutical,…etc). As the final report, the group had to submit a research proposal, based on an identified gap in our knowledge about the disease. I did prepare students by having a workshop on how to read and evaluate scientific papers, how to brainstorm scientific ideas, and explained the learning outcomes of the assignment. I encouraged students to work in the areas of they interest, as that would strengthen the group. Despite the fact that I thought it would be a very challenging task, students seemingly enjoyed it and produced high quality work and great presentations.
    The success of the assignment, something that at the time was radically new to the students (we now have many science courses the provide similar challenges) was the preparation of students to the task ahead. They understood what is expected from them and also what they can expect to gain from this exercise.

Leave a Reply

Your email address will not be published. Required fields are marked *

*