Assessment and Feedback

Day 2: Getting students ready

In Day 1 of this course, we looked at why peer assessment can be valuable, and the different types. Today, we will explore how to best prepare students to undertake peer assessment, and look at the common issues faced when bringing peer evaluation into a course.

Here are some of the issues to take into account when preparing students for peer assessment and feedback:

Group of students talking

Unfamiliarity

Peer assessment and feedback can be new experiences for some students, who may not have the pre-existing skills and knowledge to constructively engage in peer assessments and feedback.

Students can also be concerned about how relevant and meaningful peer assessment is to their overall learning.

Likewise, the lecturer or tutor may also be concerned that students will reject the idea of peer assessment and feedback due to previous experiences or pre-existing perceptions.

Social Risk

There is some social risk for students, even if it’s a low weight assessment or feedback task. For instance, the psychological impact of assessment by multiple peers can be amplified by repetition and the inability to question the assessors.

Likewise, some people may feel awkward or embarrassed in presenting to peers or sharing their work.

Student Diversity

Pencils of multiple colours

Peer assessment has been used with an enormous variety of students, however differences between students in a class can affect their readiness for peer assessment.

For instance, how can peer assessment work if students are:

  • New to the program
  • New to university
  • New to the country
  • Inexperienced in the language

How may the assessment be affected if a student is:

  • In the same long-term project team
  • From a different language, nation, culture, gender, major, or socio-economic background
  • More successful, privileged or popular
  • Related or in a household or relationship
  • A person with a disability
  • Unpleasant, disengaged or uncooperative

How can we prepare students to undertake peer assessment?

1. Create an environment of trust

Girl and horse touching heads

  • Explain the purpose of the assessment or feedback task and most importantly, highlight the benefits of peer assessment and feedback.
  • Acknowledge and plan for students’ feelings around being assessed by peers. For instance, you could provide tips and model how to give and receive constructive feedback. Read UTS’s guide for more info.
  • Give students multiple opportunities to practice first before making it assessable, and ask for feedback regularly.

Want to know more? Check out our espresso course on Fostering Student Wellbeing.

2. Create a plan

Holding map

Good planning and design will go a long way to creating a successful peer assessment process. This includes:

  • setting up clear expectations, objectives and learning outcomes of the peer assessment process
  • being explicit with the assessment criteria being used (if applicable)
  • starting with low stakes (or zero stakes) assessment. Many practitioners recommend a zero-weight practice assessment followed by guided reflection, to uncover and correct misunderstandings of procedure, and to soften student oversensitivity to feedback.

Other things to consider:

  • What assessment task and/or activities is it appropriate for?
  • Summative or formative? (i.e. Should it count towards the grade or not?)
  • Open or anonymous?
  • Use online tools?

Want to know more?

Check out the comprehensive peer assessment guides developed by University of NSW, Deakin University and University of Exeter.

question markDiscussion Questions

  1. What has your experience of being reviewed or assessed by a peer like? Has it been positive or negative (or both)? What might have helped improve your own experience?
  2. When do you think is a good time to introduce students to the concept of peer assessment and feedback in a course and/or during their study program/degree?
  3. What considerations would be most important to you when preparing for a peer assessment or feedback task?

Optional Reading

  1. Douglas Magin (2010) Reciprocity as a Source of Bias in Multiple Peer Assessment of Group Work, Studies in Higher Education, 26:1, 53-63, DOI: 10.1080/03075070020030715
  2. Rotsaert, T., Panadero, E. & Schellens, T. Eur J Psychol Educ (2018) Anonymity as an instructional scaffold in peer assessment: its effects on peer feedback quality and evolution in students’ perceptions about peer assessment skills, 33: 75. DOI: 10.1007/s10212-017-0339-8
  3. Vahid Aryadoust (2016) Gender and Academic Major Bias in Peer Assessment of Oral Presentations, Language Assessment Quarterly, 13:1, 1-24, DOI: 10.1080/15434303.2015.1133626

 

 

20 thoughts on “Day 2: Getting students ready

  1. What has your experience of being reviewed or assessed by a peer like? Has it been positive or negative (or both)? What might have helped improve your own experience?

    I’ve had mixed experiences with peer feedback. Where the peer is experienced in assessment and understands exactly how the assessment works, then the feedback is usually beneficial and constructive. However, I’ve experienced times where the feedback is either useless (very wishy washy) or destructive because the reviewer isn’t clear on what the criteria actually are or do not fully understand what they are reviewing

    When do you think is a good time to introduce students to the concept of peer assessment and feedback in a course and/or during their study program/degree?

    From what is written here, it would seem like later on in the degree (e.g. 3rd year). However, it would also seem like it does take a large amount of time to set up. Given there is limited class time (because students don’t turn up to lectures) this makes it difficult.

    What considerations would be most important to you when preparing for a peer assessment or feedback task?

    I think consistency and standardization across the peer assessment so that all assessments are reviewed in a similar way and to a similar standard

    1. Hi David. Thanks for sharing your experiences. I particularly like your examples of peer feedback, which highlight the critical role that prior experience and clear understanding of the assessment process and criteria can have in creating positive peer feedback.

  2. I would categorise my experiences of being peer-reviewed and -assessed as neutral; not negative, but certainly not positive either. As mentioned on Day 1, most of my peers only did the assessment part, and omitted the review part. I do not believe there is much value in ticking boxes which, themselves, encompass multiple facets. For instance, what does a tick for “Presentation” or “Content” actually mean? Without the review element, the assessment appears to have little value for tasks with subjectivity, such as research and presentations. There is always room for improvement and development. Unfortunately, my experiences to-date with peer assessment do not recognise this. Teaching students the importance of peer reviews, and how to give reflective feedback, would be an immense improvement.

    Related to this, I believe that peer review and assessment should be introduced to students as early as possible. Not only do these skills inform students about the challenges of the grading process, it will also help develop their own analytical and editing skills. The more review and assessment you do, the more conscious you become of your own work.

    1. Hi Bhavani, thanks for sharing your experiences. Yes, I agree that specific constructive feedback can provide really valuable learning opportunities that a simple ticking of boxes may well miss. You also made a really great point that starting peer assessment early in a student’s degree provides more time to gain skills and develop reflective practice.

  3. In my experience as a student, I find peer assessment to be a mixed bag, and a lot of “you scratch my back, I’ll scratch yours” which somewhat negates its value, as the grade becomes more important than the feedback. In my experience as a tutor, it has largely worked well, in part because the grade from the peer reviewers is minimal. This means students can focus on giving qualitative feedback, with little fear of reprisal from their peers because the grade decision is taken out of their hands. It also allows for a conversation to happen between students leading to a greater understanding of the feedback and the decision-making process of the reviewer. I hope to continue to use this model.

    1. Hi Danny – thanks for joining us! This is definitely a concern around peer assessment, particularly if it contributes to grades. In my own experience this sort of issue (students giving each other good / non-critical remarks) is more of an issue than being too negative or critical. We’ll explore this a bit more tomorrow when the post is about reliability and validity of peer assessment.

    2. Daniel, have you had to adjust grades where there was “you scratch my back, I’ll scratch yours”? I have had to do this occasionally where a student was getting an unusually high grade from an unusually small number of peer assessments.

  4. What has your experience of being reviewed or assessed by a peer like? Has it been positive or negative (or both)? What might have helped improve your own experience?
    I’ve found it generally to be a positive experience so far, but in my experience of PA, the situation was quite moderated, such that the students probably couldn’t give bad marks based on unfair reasons. That said, I feel like if I received really negative feedback from another student, I might struggle with accepting it on merits.
    I think, in that case, that I would feel better about receiving negative feedback or lower marks if there was an explanation accompanying it AND if I felt like the student was equipped to give that degree of feedback. So, I guess, as mentioned above, success in both PA and PR might require a degree of training pre-evaluation, as well as a level of respect among students for the value of each others feedback and for the sensitivity of the wording…?

    When do you think is a good time to introduce students to the concept of peer assessment and feedback in a course and/or during their study program/degree?
    I think maybe in 3rd year, or in the last year of undergrad for summative assessments and earlier e.g., 2nd year, for formative? Basically, in science at least, summative PA should be at the point where the actual skill of being able to critically evaluate other peoples work is coming together (unless there is a very solid rubric to follow that cannot be misinterpreted easily).
    If students can comprehend journal articles and understand their limitations, then they should be able to learn how to successfully peer review/assess . I guess in science students do start to achieve these skills, but I’m not sure at what point they are actively taught vs. supposed to seep into the student consciousness over time!

    What considerations would be most important to you when preparing for a peer assessment or feedback task?
    Fairness, quality, qualification, respect.

    1. Hi Angela. That’s great feedback about how moderation can help foster a positive peer assessment experience!
      I also like your comment about how the timing for introducing summative peer assessment can very context dependent, based on when students are expected to have developed certain skillsets in a specific discipline.

  5. As I mentioned in yesterday’s comment, according to my personal experience as a student, tutor, and a lecturer, the cultural aspect is very important, especially with diverse classrooms like those at ANU. People from high context cultures tend to be sensitive to peer review in general. Some cultures, where lecturers and teachers are perceived as the ultimate knowledge providers, do not even accept the idea of peer assessment. One example that is not directly related, during my master’s studies within a highly international cohort in London, one student lost the class rep election at the start of the year and didn’t show up to the class for the entire programme duration. He later told me felt so ashamed and disgraced. You can imagine what would happen if he received negative comments directly from a colleague!

    One interesting aspect, I also noticed, is that a minority of students who are new to open societies like Australia, tend to be little harsher. My personal interpretation is that these students are not used to give or receive negative comments, and when they experience the open feedback atmosphere, they tend to overdo it and possibly give too harsh comments. I personally used to be on of these students several years ago, before adjusting to what is and is not acceptable 🙂

  6. At first I was a little resentful at being peer reviewed: why was I paying a lot of money to have someone who didn’t know any more than I did tell me what to do? What changed my mind was doing the peer reviewing myself: I saw deficiencies in the other students work which I realized were in my work.

    It would have helped to have the peer review nearer the beginning of the program, with some small, low stakes assessment, before the capstone which decided if I got a degree, or not.

    Also it would be useful for the students to introduce themselves to the class, perhaps using one of those icebreakers where each student introduces another: “This is Tom he is a …”.

    Most important, I suggest, is to reassure the student that the peer work will be reviewed by an instructor.

  7. The only direct experience of peer assessment or review I’ve had has been that of peer review of proposals or papers, which is not particularly relevant because this is such a high stakes process. But what I have learnt from this process of peer review is that feedback can vary widely from one reviewer to another, and that the degree to which a review is useful depends on the way that feedback is given. But looking back to my undergraduate years, I can imagine that I would have found the idea of peer assessment initially quite confronting because I wouldn’t have wanted to say anything that upset another student. I probably would have also worried about the fairness aspect of it. However, I can also imagine classes where I think peer assessment would have worked well and where I would have comfortable expressing my opinions and listening to feedback from other students. These were the classes where the teacher was good at fostering a supportive environment and collaboration between students, and clear about what the criteria for assessment were.

    When should PA be introduced? I saw Tom’s comment about how it would have been better if it had been introduced earlier, with low stakes. From the reading above and from what others have written, this makes sense to me. It would be good to introduce it around second year, with formative and informative tasks so that students have the opportunity to learn how to do it in a non-threatening environment.

    But whenever it is used in a class, I think you have to assume that it’s new to at least some of the students so go over the principles of it with them and attempt to establish a type of class contract about respectful interaction, as well as clearly outlining the assessment criteria. As Zohair writes, we have a diverse student population with widely varying background and experiences. Discussions about expectations not just of the work itself but also of the process of peer feedback and assessment would hopefully help create a more inclusive atmosphere. One of the readings even suggested incorporating a discussion about bias in grading for example.

  8. I have had both positive and negative experiences of peer review. I am quite comfortable when the feedback is anonymous and delivered when I have time and space to digest it as objectively as possible. In contrast, when feedback has been delivered publicly (ie. In tutorials) by the person assessing and there is no space to consider it properly, it makes me quite uncomfortable and I find it difficult to concentrate on anything constructive that may have been offered.

    I think that peer assessment should take place a little way in to a degree or program, when students are used to their environment and their student cohort, and have attained at least a little subject knowledge and so have a wider frame of reference. The considerations that would be most important to me if I were preparing a peer assessment task would be reading the cohort dynamic, and trying to create an atmosphere of support. I would also want to explicitly discuss the purpose and value of feedback (distinct to a mark or grade), consider how and when students received their feedback and, perhaps more importantly, what they did with it after the task was complete.

  9. I think peer review is a very important skill for students to learn. As we grow and develop, we no longer have teachers to rely on, so we use peers to help us improve. Isn’t that what we do for journal article peer review?
    Because it’s a skill best developed through practice, I think it can be introduced quite early in a student’s career, low stakes and with a lot of scaffolding to start with.

  10. What has your experience of being reviewed or assessed by a peer like? Has it been positive or negative (or both)? What might have helped improve your own experience?

    Generally, I have had good experiences of peer review in classes, even if at the time I didn’t necessarily understand why we were doing it. I was less nervous to read feedback from a large group of people, partly because I developed an understanding that assessment is subjective, so the comments could vary wildly. On the other hand, if a comment was recurring, then I knew it needed addressing.

    When do you think is a good time to introduce students to the concept of peer assessment and feedback in a course and/or during their study program/degree?

    I like the idea of introducing it reasonably early in a degree, and early on in a course. Both give students a better idea of how the marking and assessment will work, but also how they can improve their own work. In yesterday’s course, the video mentioned introducing peer assessment in second year, and I think that is a good idea for many students. It gives a chance to adjust to uni life, and get some feedback on a baseline of work, before going into a feedback process like this one.

    What considerations would be most important to you when preparing for a peer assessment or feedback task?

    I think the biggest consideration for me is being really clear and explicit about the aims, the goals, the reasons, and the expectations in any kind of assessment task. If students know what they’re supposed to learn from the task, they are more likely to appreciate it when they can do it, and more likely to be fair in their assessments of others.

  11. My main experience of peer review is peer assessments on submitted papers and this experience has ranged from wildly different reviews to uncannily harmonious ones. This range is exactly what we could expect in a classroom too. On conference organising committees I’ve also been surprised by the range of approaches to assessing presentation abstracts and the range of evaluations for one abstract (and this is by assessment specialists!!). The issue of cultural expectations of assessment and public critique (even if ‘public’ is just 2 people) is a really important one with students of increasingly diverse cultural, linguistic and educational backgrounds in our classes. We can no longer assume our students are comfortable with anglo-western classroom cultures. When I’ve used peer feedback, I always give the option NOT to give or receive feedback, and any feedback given is acknowledged by the feedback giver. It’s my effort to respect different types of engagement and to engender respectful feedback giving.

  12. I feel the psychological impacts of assessments are amplified in peer assessments than “traditional” assessments. It was not caused by repetition, as mentioned in the material, but the relationship of “peers”. So the feedback is from someone of a closer relationship instead of a lecturer or tutor with more authority. I would be more delighted from a positive feedback and more depressed from a negative one, even though the feedback might be not informative at all. The reasons might be cultural background and pre-existing experience.
    An informative feedback will definitely improve my own experience. It should include clear marking criteria and practical ways to develop the work. The goal is not only to get a positive feedback but to really learn something.

  13. My experience of being reviewed or assessed by a peer was very different when I was an undergraduate student as to when I was already a working professional. My maturity as a student affected or influenced how I viewed the feedback and assessment from my peers (which Russell W touched on, on his post in Day 1).

    As a young undergraduate student, I’d say that the experience was neutral – not positive, not negative, even if I felt very, very uncomfortable every time we did it. And I was not entirely sure if I was being given honest feedback or some of them may be holding back. There were also times when I received negative feedback and I would think – who are you to tell me that?! Similarly on my part, I would filter and/or edit my feedback, mainly doing what Daniel K mentioned in his post as “you scratch my back, I’ll scratch yours”. I still found peer assessment stressful and confronting in my more mature years as a student but I’ve come to appreciate the feedback more, seeing it as an opportunity to improve my work.

    I like the idea of introducing peer assessment as a formative task in some small, low stakes assessment as Tom W mentioned. Perhaps if someone highlighted the benefits of peer assessment and feedback way back, I would have appreciated it better. I remember we thought we were being made to do it because our teacher was just being lazy. It’s also a good idea, as mentioned by a lot of the other participants, to do it in later years, when students are more familiar with their environment and peers and a degree of trust has already been established.

  14. What considerations would be most important to you when preparing for a peer assessment or feedback task?
    >> As foreshadowed above, trust and predictability within the cohort are two conditions that would have to be met before introducing either assessment or feedback among peers. Building trust is mainly a question of time, familiarity and of course teaching staff setting the behavioural example by demonstrating how respectful debate and open sharing of views takes place. On predictability, what I mean by this, is that students must have a sense of the process of feedback/assessment (the different steps involved) but also a very clear understanding of the role of the process and how it will impact student grades and assessment. I also have an intuition that perhaps this might be an opt-in exercise to begin with – to see how many students would engage reciprocally in the activity, eg assess and be assessed – and for those students who do not want to take part, having a second option (eg, tutors or TAs from other courses doing the peer assessment or something like that). Perhaps most would opt-in to the peer approach, and by doing so, would already have bought in to the process before it even formally started.

  15. 1. What has your experience of being reviewed or assessed by a peer like? Has it been positive or negative (or both)? What might have helped improve your own experience?
    i have only had peer review of papers and grant proposals, but never really any assessments (much like Sally pointed out above). These are a bit of a mixed bag, some are constructive and useful and some not. For me personally, is the mindset that i bring to the table when reviewing such feedback. I have been told in relation to grant proposals to read the reviews and put it aside for a few days, just to process it. I find that a very useful tip!
    2. When do you think is a good time to introduce students to the concept of peer assessment and feedback in a course and/or during their study program/degree?
    I only teach postgraduate Masters students. Many of the domestic undergrads have had exposure to this sort of assessment so its really easy for them to take it on board. we have a majority of international students and for them it is really hard. I have started a peer mentoring trick when they are encouraged to ask questions related to any of this from their peers. Setting expectations early on is key. they have access to the rubrics, the process (how to record a narrative and upload it, how to use the workshop tool). I also have the Academic Skills and Learning center do a session on presentations using a previous students work ( with permission) as an example which they have found useful.
    3. What considerations would be most important to you when preparing for a peer assessment or feedback task?
    Given i have just finished teaching for this semester, there are a few things i can improve on. One easy win is process- giving them absolutely every little tiny bit of information they need so the process is not a worry. Something i need to work on! I mentioned in my post for Day 1, i was blown away by the high quality interaction between the online and in-person students over this task. I realise it was assessed and maybe that is why but it was just very good to see and be a part of.

Leave a Reply to Rebekka Cancel reply

Your email address will not be published. Required fields are marked *

*