Assessment and Feedback

Day 4: How to make feedback effective

Written by our Guest Facilitators, Dr Olga Kozar and Geraldine Timmins from Macquarie University

As mentioned in Day 1, according to a large synthesis of educational studies (Hattie,1999), effective feedback gives learners cues on not only what they could do differently, but also how they could do it.

Or, using wording from Feedback for Learning (a study of feedback designs and conditions to guide educators, academic developers, instructional designers, and institutional policy), “the information provided is usable, and learners know how to use it”. As discussed in Day 3, feedback is best coming at a time when students can still use it or will, at least, hear it.

In this post, we’ll dive deeper into making feedback more usable and prompting more action from students.

Tip 1: Pick your battles and name the main ones first

When we give feedback, we often start with the points that first jumped at us. Sometimes these ‘first’, or most obvious points happen to be the most important ones, but often aren’t. Students, on the other hand, tend to perceive what we say first as as the most important information (the primacy effect). They also remember the last point well (recency effect).

Image: http://learnlanglab.com/

So if you are not doing it already, it may be a good idea to arrange your comments in the order of importance, and focus on those points that have the biggest impact for the students.

How to:

When arranging written feedback, you may have more time to sort your feedback points in order of importance, however, in tutorials, when giving verbal feedback, it may come out as stream of consciousness, and therefore may be influenced by the primacy effect.

So if a student approaches you for feedback during a tutorial, you can always ask the student for some ‘time-out’ to arrange your feedback response, or you can provide feedback in any order and ask the student what they think the most important points are. This discussion will not only provide valuable opportunities to clarify to the student(s) how they should invest their time/efforts in taking action on the feedback, but will help you assess whether the student has really heard you.

Tip 2: Take advantage of audio and/or video technology

Interesting stats: Lunt & Curran (2010) report that students in their study were 10 times more likely to open/listen to audio feedback.

Image: By Wes Hicks, Unsplash.com

According to a growing body of studies looking at audio feedback in tertiary education (see for example, Ice, Curtis, Phillips, & Wells, 2007; Lunt & Curran, 2010), hearing feedback can have a bigger impact than reading feedback comments. It may be that audio allows educators to convey more by using intonation and other verbal cues.

It may also be that speaking is generally faster than writing, and audio feedback can potentially ‘pack more information’ than written comments, and therefore contain more ‘actionable’ points, or it may be due to other reasons.

Where there was audio, there is video now…

Fast forward several years, and screencast/desktop capture technology is now widespread enough to be available to tertiary educators, and many are experimenting with video feedback. In their review of 37 studies of video feedback, Mahoney, Macfarlane, & Ajjawi (2018) report increased student engagement and even (surprisingly!) time saving by educators when giving feedback. Students also commented that video feedback helped them avoid misinterpretations and motivated students to act on comments more than written feedback would have.

So it looks like video (or at least audio) feedback may be worth a shot (bearing in mind there will be an initial learning curve and some time investment in set up and familiarisation). However, this investment is likely to pay off, and will save you time longer term.

How to:

Feedback Studio in Turnitin – You can record ‘voice comments’ when marking in Turnitin. Access a step-by-step guide on this function here.

Speaking of Feedback Studio and saving time…

Did you know that you can save comments and even share them with other colleagues? Having some well-written and detailed comments on common topics (QuickMarks) that you can easily drag and drop will help you save time and focus on individual feedback.

VoiceThread – VoiceThread (available at MQ) allows you to add audio or video comments to media, such as video presentations, visual portfolios or voice recordings (e.g. language aural exams, visual media assignments etc.) at specific timestamps or by pointing out specific areas of the visual field.

Personal Capture – Echo360’s Personal Capture software allows users to record and edit screencasts/desktop capture, so you can easily create a short video-feedback recording. More on Echo360 PCAP here.

Tip 3: Work on students’ assessment literacy and feedback involvement

Improving learners’ feedback literacy (Sutton, 2012) goes a long way. It’s important to help students shift their perception from ‘feedback is a hopes crusher’ to ‘feedback is a useful tool’, and encourage students to be proactive in seeking feedback.

How to:

  • Try having ‘assessment dialogues’ (Carless, 2006) and being explicit about the ‘rules of the assessment game’. Have a discussion in class on what is valued in your discipline and what is expected from this type of assignment. E.g. What do rubrics mean? What level of understanding and competence do you need to demonstrate to achieve a high distinction as opposed to a distinction or credit? How will this knowledge be useful in a workplace setting within this field?
  • Give students a chance to assess and give feedback. Research suggests that learners often gain more from giving compared to receiving feedback (Nicol, et al, 2014). Why not provide students with an opportunity to assess and give comments on the work of their peers?
  • Encourage students to ‘nominate’ areas they want feedback on. This turns them from ‘passive’ feedback receivers to active feedback seekers, which could make them more open to hear and act upon the information they receive.
  • Require feedback diaries and ‘action plans’ and introduce them in assessments.
  • When providing feedback, listen. The student may give you a clue to what they are not understanding, areas they feel they could make improvements on and what more they may need from you.

In summary, improving feedback effectiveness requires not only changes to how we provide feedback, both in the content, (e.g. more specific feedback), and the delivery method, (e.g. audio feedback), but also in how we carve out opportunities for students to self-assess and give and receive peer feedback.

question markDiscussion Questions:

  1. Have you tried any of the ideas mentioned above in your teaching? What worked? What didn’t? Why?
  2. What are some possible issues you might encounter from using these techniques? How might you address them?
  3. Are there any other tips you could share with our community?

Further Reading

  • Macquarie University’s Feedback Guidelines for Staff
  • Macquarie University’s Effective Feedback Resource
  • Ice, P., Curtis, R., Phillips, P., & Wells, J. (2007). Using asynchronous audio feedback to enhance teaching presence and students’ sense of community. Journal of Asynchronous Learning Networks, 11(2), 3-25.
  • Lunt, T., & Curran, J. (2010). ‘Are you listening please?’ The advantages of electronic audio feedback compared to written feedback. Assessment & Evaluation in Higher Education, 35(7), 759-769. doi:10.1080/02602930902977772.
  • Mahoney, P., Macfarlane, S., & Ajjawi, R. (2018). A qualitative synthesis of video feedback in higher education. Teaching in higher education, 1-23. doi:10.1080/13562517.2018.1471457.Sutton, P. (2012). Conceptualizing feedback literacy: Knowing, being, and acting. Innovations in Education and Teaching International, 49(1), 31-40.

 

 

 

16 thoughts on “Day 4: How to make feedback effective

  1. I didn’t know about the Turnitin voice comments- as a French teacher, I will definitely be using this function to add in further learning (i.e. listening) opportunities!

  2. The “probability of remembering” diagram confirmed some of my own experience and observations. I also appreciate the idea that the process of assessment should be a “dialogue” instead of a “monologue”. As an extremely camera-shy person, I need some time to get accustomed to these audio and video feedback. But I’d like to recommend other coffee courses under Module 3 (Delivering Content Online), which contain many great suggestions on video shooting.

    1. Thank you, Sunny. Maybe start with the audio (not video) feedback first? Research shows that, yes, there is a learning curve, but once you get used to it, it’ll save you time, and you’ll be able to provide richer (and more interpersonal, hence, more effective) feedback.

      1. Olga, I don’t see how audio comments would save time, unless there is some form of audio templates. At present I will select standard prepared comments for a student. Can this be done with audio? That is can I tell the system to insert audio clips 3, 6, and 8 for a student and not have to say the same things over and over again?

        Also as the student recipient I would have to listen to the comments dozens of times to be able to make use of the comments. I might even need to transcribe them. Much of the time when being given live verbal feedback I have no idea what the instructor is talking about. Even when I do know I am so annoyed by what is said I miss what is said next.

        I question the usefulness of providing richer feedback. The student has only a limited capacity to accept feedback and if we provide too much they will be overwhelmed. Also I question the ethics of pretending to provide personal customized feedback if not resourced to do this. Where I have one student doing an individual project and I devote an hour a week to them, they do get personal attention. But in a conventional class a student will be lucky to get a few minutes personal attention a week. I suggest we need to be honest with a student and explain their learning is mostly in their hands and we can only provide limited guidance.

        1. Hi Tom,

          I share many of your concerns about audio/video comments. A recent conference held on campus spruiked the benefits of using voice memos to provide feedback. Unfortunately, these concerns seemed to be a minority view and were roundly dismissed by the presenter and audience.

  3. I’m usually selective in giving feedback. I try to focus on what I perceive as the most important issue, rather than adding a lot of comments on different issues. This is fairly easy to do when giving written comments, and for general oral feedback if I’ve had time to prepare in advance.

    I like the idea of asking the students what they perceive to be the most important feedback when we are having a conversation. I’ve done this sometimes if the student looks confused or unfocused, but it is probably a good idea to add to the routine for every feedback conversation. Another question that also works here is to ask the student what they think they could do differently next time, or what their next steps for an assignment will be.

  4. I now use the selective approach, but it took me years to come to terms with more feedback not being BETTER feedback.

    What I do now is make written comments in margins of an assignment in the traditional way. Most of the comments inserted come from a template. I then copy the most important comments to the marking rubric. The student then sees what they got for each criterion of the rubric, with a comment next to it. I also copy these comments to the field provided by the marking system, so they are displayed next to the mark. This way the student sees a mark with comments. If they want they can then look at the rubric with more comments and then the paper itself, with details.

    I have not tried audio recorded comments, as I worry about the amount of time this would require and the risk of being misunderstood. For accessibility reasons any audio comments also have to be provided in written form anyway. For the same reason I don’t provide students with face-to-face verbal comments.

  5. I found the probaility of remembering graph very interesting. I have always tried to start my feedback with positive comments, to act as ice-breakers and reassure student before telling them what they can improve on. However, this suggests that the most important comments get lost if they are in the middle. I think I will need to think about how best to apporach this next time I am providing feeback.
    I also like the idea of asking student what they think is the most important points.

  6. I am improving the method of giving written feedback in terms of ordering my points by importance. It was a recent discovery that through a feedback of feedback, a student discussed the first two points of my comments, not mentioning the most important last point.
    I will definitely try audio feedback this semester, as I do prefer something more personal. And I would encourage students to ask question in person or send me an audio file.
    While giving feedback, I believe pointing out the strength before the areas need improvement would put students in a better position to accept the feedback and not make the feedback as a “crasher”.
    Something I would improve is about student’s assessment literacy. They don’t understand that tutorial quizzes are a form of feedback. The conversation we have is form of feedback. I will highlight this in my own course survey. Let’s see how I go with it this semester.

    1. Dear Hua,
      How did you go with the audio feedback? Would love to know how it went!
      I think that audio and video feedback is generally underused. I was once helping a student who was a professional in her country and was going to sit English tests to enable her to practice here. I recorded us practicing a workplace scenario then we listened back (to the tape) and I paused to give her encouragement and suggestions then we practiced another scenario. Thinking back on that, it would have been a lot for the student to take in so would have been better to record my spoken feedback as well for the student to listen to as many times as she needed to feel more confident (if only I’d had a smartphone at the time!).

      Screencapture video feedback does sound like it would be quite time-consuming but I’m sure it is really useful for students. Unlike plain audio, teachers can point with their cursors to show to exactly what they are talking about as they speak either in a document or paused audio/video. I like to use the very reasonably priced https://www.faststone.org/FSCaptureDetail.htm (it also doubles as a handy screenshot tool far superior to the Snipping Tool). If you are looking for something free, there are online tools like Screen-Cast-O-Matic which can record short watermarked videos but I if you are going to comment on audio or video, make sure you can record the system audio so students know can hear what you have just heard.

  7. I already implement assessment dialogues and selective feedback. Regarding the latter, I find providing individualised comments to students, and a general feedback sheet to the whole class saves time (as I am not repeating the same feedback over and over), helps students identify their key areas for improvement, yet ensures they don’t miss out on the rest of the feedback. This approach has been well-received by students who have a positive approach to feedback. The students who choose to ignore feedback ignore both sets of feedback.

    In terms of assessment dialogues, most of mine tend to be about shifting the focus away from “what does Prof X want?” to “what skills/knowledge is this assessment trying to achieve?” I have found that students attempt to tailor responses according to who they think will be marking the assessment, rather than focusing on the relevant learning outcomes. It is difficult trying to convince students to make this shift. At some point, the pleasing/anticipating-the-marker philosophy seems to get ingrained in students, and any other suggestions are viewed with suspicion. One way I have tried to address this is by double-marking work with my co-convenor. It is time-consuming, and I really couldn’t say whether it is convincing students to change their approach to assignments. The one thing I have noticed, however, is that it appears to be the same students who have a negative approach to assessments and feedback, who, in turn, do not engage with feedback, and thus, try to please-the-marker in order to get the easy grade rather than work on improving themselves.

    I have occasionally had students ask me to point out/look for certain things when giving feedback, and have usually been happy to oblige. I am keen to try rolling this out more widely, and incorporating nominated feedback areas. To streamline the process, I may consider getting students to select from a list of limited options.

  8. I thought the primacy/probability of remembering framework resonated – in a related way I have had students admit to me that if they get lots of negative feedback they just don’t read any more. As such I tend to limit my feedback to bigger chunks and try and make them constructive – I don’t necessarily do the famous feedback sandwich as students say they see straight through it (although I do think negative feedback needs some cushioning) but I love the COIN method which I think I heard from Glen O’Grady from CHELT – if memory serves me correctly, it stands for Context, Outcome, Impact, Next. Basically, you frame your feedback as What the student was asked to do, what they actually did, what the impact of that was and how they can do it differently next time. I feel like this really helps you be specific with your feedback and make it seem like it isn’t personal to the student, particularly useful when you have to give quite negative feedback.
    I haven’t tried to do the voice feedback, but a I have heard elsewhere that if a student thinks the marker knows who they are they are more likely to engage with the feedback (an interesting fly in the ointment for anonymous marking) so I always start my comments with Hi Jane etc to reinforce the personal relationship, and I guess an audio approach could do the same thing.

  9. I agree that feedback needs to be action oriented. General comments seem to confuse some students and these students do not know how to translate it into a change for improvement.

    I have heard positive things are the audio or video feedback from both tutors and students. I would like to encourage my tutors to trial this in 2020. I also like the use of QuickMarks. I have found similarity in what I am looking for in quality assignment content and what students do well or need to improve on. Therefore, QuickMarks is useful to target these key areas. These can be edited so the comment can be further personalised.

  10. I am quite intrigued by the voice feedback function in Turnitin. I am sure we have all experienced the fact that tone is often easy to misinterpret in written feedback, so this seems like a good way around that. It also might help bring back a greater sense of connection to online learning. I would be interested in hearing from anyone who has used this? Did you use only voice feedback or did you also provide written feedback as well? I am concerned that providing only voice feedback could have some accessibility problems and might make it more difficult for students who prefer to re and reread feedback. An option would, of course, be to provide both but this would then make it more time intensive rather than less.

Leave a Reply to Emmaline Cancel reply

Your email address will not be published. Required fields are marked *

*