Assessment and FeedbackCoffee Course

Day 1: Why is feedback so tricky to get right?

Written by our Guest Facilitators, Dr Olga Kozar and Geraldine Timmins from Macquarie University

Feedback is the information students receive about their learning/performance which can, if handled well, promote development and improvement.

Feedback can have a potent impact on student learning – according to a large synthesis of 180,000 educational studies (see Hattie & Timperley, 2007), feedback is among the top influencers on student achievement.

However, a recent study found that despite its potential to be powerful, in reality, feedback is rarely powerful. Research consistently shows that university students often find the feedback they receive unclear, untimely or inconsistent (Hounsell 2007; Nicol 2010).

Reflection Question:

Will and the funhouse mirror, Eric Chan, CC Licence Attribution 2.0

What do you think are the main reasons why feedback doesn’t reach its full potential? In other words, why is it that feedback can be powerful, but rarely is powerful?

Post your thoughts in the comments section.

Misconceptions about the feedback experience

Before we get to the ways in which we can create a better feedback environment, it might be helpful to clear away some misconceptions. We have collected 5 common misconceptions about the feedback experience – however, this is not an exhaustive list, feel free to share any others in the comments section!

Misconception 1: The role of feedback is to justify the mark

Feedback is often viewed as justifying the marks – ‘you got a ‘pass’ because…’, ‘you didn’t get a HD because…’.

However, good feedback can and should do more than issuing a ‘post-mortem’. Feedback’s real power is to be an instrument for improvement (formative feedback rather than summative feedback), and good feedback should give students explicit tips on WHAT they could do differently.

Ineffective practice Effective practice
Feedback only justifies the mark. Feedback provides explicit and actionable tips on how students can improve their future performance (e.g. in the next assignment).

 

Creating a learning environment rich in useful and usable feedback is even more important today because of the changing nature of education in the time of sudden information abundance. However, this information bonanza has not taken away the need for university education. Quite the opposite, personal feedback and attention is what gives higher education its real value.

Misconception 2: The educator is the only source of feedback

Traditionally, educators were viewed as the main ‘feedback providers’, however expecting educators to provide all the feedback to students is not only unsustainable, but it wastes valuable learning opportunities.

Research shows (Boud & Falchikov, 2007; Li, Liu, & Steckelberg, 2010) that self-evaluation, reflection and peer-feedback are powerful tools for development (often even more effective than the educators’ feedback). Educators could shift from being ‘feedback providers’ to ‘feedback opportunities creators’.

Unsustainable practice Good practice
The educator is the only ‘feedback provider’ The educator is a ‘feedback engineer’ and creates opportunities for peer feedback, reflection, self-assessment.


Misconception 3: Students recognise when they receive feedback

 

We have a colleague, let’s call her Anna (not her real name), who received comments on the anonymous student survey that she was not giving enough feedback.

Anna was gobsmacked: as she thought she was giving a lot of feedback.Students may have confused her feedback with ‘general comments’, especially when she was giving feedback to the whole class and interspersed feedback with other information.

Anna then made her feedback comments more explicit by saying things like ‘Here’s my feedback on …’  or ‘Here are some things you can do to improve in the future…”. While it felt awkward, it helped her students identify what was feedback among other information.

Ineffective practice Effective practice
Feedback is ‘hidden’ among other information, and students can’t recognise it. Feedback is explicit and the students’ attention is drawn to the fact that they’re  receiving feedback.

question markReflection Questions

  1. How do you give feedback to students?
  2. Do you think your students recognise when you’re giving feedback?

Post your thoughts in the comments section.

Misconception 4: Students understand and receive my feedback

An interesting study by Chanock (2000) explored whether students misinterpret feedback. The researchers took a common comment “too much description; not enough analysis”, and asked tutors and students what they believed the comment meant. The study found that “almost a half of the students who responded did not interpret this comment the way their tutors had intended it” (p.95).

This illustrates the need to use clear details and examples when communicating with students. We’ll look at this more on Day 4.

Ineffective practice Effective practice
Feedback uses vague or broad academic terms Feedback explains academic terms, uses plain language, gives details and clear examples.

Misconception 5: It’s not possible to provide quality feedback in a short amount of time

Due to large student cohorts and casualisation of teaching, markers are often faced with a tight time allocation, e.g. 15-20 minutes per assignment.

While it is certainly challenging to provide quality feedback in a short time, technology might be able to help.

An increasing number of studies (see Mahoney, Macfarlane, & Ajjawi, 2018) suggest audio or video feedback can be a considerable time-saver (once educators become comfortable with the technology), and can also be more interpersonal and impactful. We’ll look at some examples of this on Day 4.

We’ll also share more time-saving hacks throughout the rest of the course, and we encourage you to share your own thoughts and tips.

The devastating effect of not receiving feedback

Lack of feedback can have serious consequences for students. Many students have no idea they’re off track until it is pointed out to them or it’s too late. We’ll share a personal story from one of the facilitators to illustrate this point in Day 3.

In summary

While literature is teeming with examples of the characteristics that make up good feedback, we believe that at the very core, good feedback needs to do 2 things:

(1) be clear to students and

(2) encourage students to take specific action(s) or do things differently.

Good feedback tells students not only WHAT they need to improve, but suggests HOW it can be done.

question markDiscussion Questions:

Think of a time when you received feedback – it may have been on your research article, grant proposal, HEA application, etc.

What made the feedback useful or less useful?

Was the feedback directed at:

  • the task (e.g. how well you’ve done something),
  • the process (e.g. strategies or approach)
  • a particular personal trait or anything else?

Share your thoughts in the comments section.

Join us for a hands-on workshop with our facilitators!

Tuesday 28 August 1.30 – 3.00pm

This interactive workshop on “effective feedback”, will guide participants through scenarios of verbal and written feedback, invite the group to collaborate on ‘sustainable’ and good practice examples, and create individual action plans.

Afternoon tea will be provided. Space is limited, registrations are essential for booking and catering purposes.

Go here for more info or to register.

References:

Chanock, K. (2000). Comments on Essays: Do students understand what tutors write? Teaching in higher education, 5(1), 95-105. doi:10.1080/135625100114984

Falchikov, N., & Boud, D. (2007). Assessment and emotion. In N. Falchikov & D. Boud (Eds.), Rethinking assessment in higher education: Learning for the longer term (pp. 144-158).

Hattie, J., & Timperley, H. (2007). The power of feedback. Review of educational research, 77(1), 81-112.

Hounsell, D. (2007). Towards more sustainable feedback to students. Rethinking assessment in higher education, 101-113.

Li, L., Liu, X., & Steckelberg, A. L. (2010). Assessor or assessee: How student learning improves by giving and receiving peer feedback. British journal of educational technology, 41(3), 525-536.

Mahoney, P., Macfarlane, S., & Ajjawi, R. (2018). A qualitative synthesis of video feedback in higher education. Teaching in higher education, 1-23. doi:10.1080/13562517.2018.1471457.

Nicol, D. (2010). From monologue to dialogue: improving written feedback processes in mass higher education. Assessment & Evaluation in Higher Education, 35(5), 501-517.

42 thoughts on “Day 1: Why is feedback so tricky to get right?

  1. What do you think are the main reasons why feedback doesn’t reach its full potential? In other words, why is it that feedback can be powerful, but rarely is powerful?
    Feedback is tightly linked to marks and students often ask “why did I lose this mark?”. Feedback should provide guidance for students to develop further. However, this needs to be coupled with students being able to act on the feedback. Too often within a 12 week course there are a range of distinct assessments with little opportunity to build on the feedback from previous work.

    1. Thank you, Matt. You write “Feedback is tightly linked to marks…” , which does sound like the ‘justification mindset’ that leads down to the ‘ justifying the mark ‘ path we’ve discussed above. What do you think could be done to change this mindset and focus more on formative feedback? Also, you rightly point out that “Too often within a 12 week course there are a range of distinct assessments with little opportunity to build on the feedback from previous work.” Again, is that the ideal scenario? What could be done differently?

    2. Matt, to allow for feedback I have assignments build on each other. However, some students have difficulty with this concept. Some need repeated reassurance it is okay to reuse their work. Some decide halfway through to abandon what they were doing and start again. Others have the mindset they will only start work shortly before an assignment. This makes it difficult for them to make use of feedback. To counter this I explicitly divide assignments into two parts, with feedback on the first part for the second.

  2. To assist learning, feedback has to be timely and the student has to be receptive to it. Telling a student what they should have done in an assignment after the marks are finalized can’t help them improve their mark and will likely be ignored. The feedback has to be clear as to what it is about and has to be actionable.

    1. Indeed! These will be the topics of our Days 2 (emotions/ being receptive to feedback) and Day 4 (making feedback effective). Thank you, Tom!

  3. I provide students with a weekly mark and one or two sentences of feedback in an individual message via the learning management system. I think the students recognize this as feedback, as I tell them at the start of the course this is what they will get and repeat it in a weekly post to all students. I put the feedback just under their mark for the week, so they are more likely to see it. I get replies from some students discussing the feedback.

      1. Olga, my students get a weekly mark for answering two or three questions in an online forum, and replying to answers from other students. I used to do the marking myself but changed to peer assessment. The students get the average mark from their peers (after any manual adjustment I might make), plus the feedback from me.

        1. What a great (and sustainable) way to provide students with feedback. I really like the peer marking/feedback idea.

  4. I can really relate to this whole list of misconceptions, in one way or another, from both a teaching and learning perspective. I really like this comment “Good feedback tells students not only WHAT they need to improve, but suggests HOW it can be done.” I think I am pretty good at conveying the “what” but upon reflection could do better on the “how”. The “how” aspect tends to be left out of a lot of written feedback in my experience. A good aspect for me to improve upon I think.

  5. The worst feedback I received was on a reflection exercise: my instructor laughed, because I had included formal references. I found this humiliating. It would have been okay to direct the feedback at the task, to simply point out references were not to be used. It would have been useful to suggest a process for writing in a scholarly way without formal references (which had been required for all work up to that point). But this was more like a personal attack on me, at least that is how it felt at the time. I learned from this to be careful not to mock my students, no matter how inept their attempts might seem.

    1. Thank you for sharing this story, Tom. We sometimes forget how emotional feedback can be and how careful we need to be with it.

  6. Hi Everybody,
    I echo what other people say about feedback and marks. One thing I really hate (and I’m open to starting some kind of petition/open letter to see if we can get this changed) is that things such as turnitin/gradebook allow you to see the mark before the feedback. By seeing the mark first many students then see the feedback as being some kind of justification or if the mark is not what they want they are looking to argue with the feedback.

    1. Hi David,
      There might be a way to hide the mark in Turnitin. I’ll let my more technically-savvy colleagues reply to this!

      1. Oh great question! I have checked with our Turnitin experts here and it doesn’t look like it is possible to hide the grade and show the feedback. Sorry David! Might need to start the petition and send it to Turnitin!

  7. I think one of the challenges is figuring out the types of feedback that each student needs and responds well to. I know I sometimes fall into the trap of thinking of the class as a group rather than thinking about each individual student or I make assumptions that most students learn the same way I do and so need the same types of feedback that I would need, but this is often not the case. For sure this is more difficult with undergraduate classes but with HDRstudents or more tailored training programs one option is to discuss types of feedback with students at the beginning of the process. In my own learning I definitely identify with the ‘no clear solution’ problem, often students already have an inkling what is wrong but are not sure about how to go about fixing it. I also think one thing that can help to get over the problemof having distinct assignments across the 12 week semester is to include content or assignment specific feedback as well as more ‘general skills’ type feedback. Some of the most empowering feedback I have ever received was pointing out some flaws or habits in my general writing style. This is something that is helpful to students across all assignments.

  8. Thanks for the interesting start to this series. However, I think you missed one important Misconception: that feedback is only of value to the student – Academics can learn about their own teaching strengths and weaknesses from the feedback too. I have a system where feedback is emailed to the students (with a grade, and usually before the “mark” appears on Wattle). The feedback itself is stored and I can review it to see if I have had to make the same suggestions or even complements to others in the course, or even during previous years. If I am “continually” giving the same advice on how to improve something in an assignment, then that is a strong indicator that my teaching or supportive opportunities need to be adjusted. I can then compare the year after making any changes …and hope that the feedback goes from “You need to think more about your Report Title as often this is the only thing a potential reader might look at and your Title a bit too general or even bland to attract any readers” through to a comment like “Great start with a concise title that still manages to convey the critical boundaries of your report”.

    1. I love the idea of sending feedback before the mark, Cristopher. Could you tell us more how you’ve arranged it? Do you do it manually or is it automated? What tools do you use?
      And, yes, absolutely! We’ve missed this important Misconception!

      1. Greetings. My “system” predates Wattle was the basis of my Teaching Award many years ago. It involves an Access database or criteria and feedback with the feedback indexed to a number 0-100 for each criteria. So, for example 0 = “No title, but you need a title because…” while 100 = “Great title…”. These numbers can be weighted (0 if you don’t want these comments to relate to the final mark, and large if they contribute lots). So feedback is directly related to marks. I mark the assessment with the Access database open and at the end have all the components and feedback for students in one place which I use for individual mail merge emails, then a couple of days later upload marks to the Wattle grade book. Happy to demonstrate to anyone interested.

        1. Hi Cris – wow that sounds like a really interesting strategy to provide a substantial amount of feedback, and quickly! I’d love to hear more about it or see how it works! 🙂

          1. Anyone else interested in a demonstration or hands-on with my Access-based Student Feeddback Evalution system? Happy to oblige:-)

    2. Hi Cris! I just wanted to echo this – I certainly have learned a lot about my teaching approach, style, etc from the types of feedback I have to give to students on their work. Great point!

  9. Reading the first question in this post – why is it that feedback can be powerful, but rarely is powerful – my immediate response was that it is a question of time. As lecturers and tutors we have limited time, a large number of assessments to mark, and often end up repeating the same general advice to students. I know feedback can be important, and I want to provide helpful feedback to my students. Yet I have often found when faced with what seems an endless number of assessments, that I just want to get it done, and so end up writing something vague and general. I would therefore be interested to hear the promised suggestions of time-saving hacks!

    The point raised about how the educator is not the only source of feedback is interesting, and one I’ve become more aware of after moving to teach at an Australian university. I think there is untapped potential here that we are not using at the ANU, compared to my experience studying and teaching at European universities before.

    One example: For the past few semesters I’ve been convening the master thesis course in my department, and as part of this I’ve organised workshops each semester where the students get a chance to present their own drafts and then receive comments from the other students and from me. I believe this can be beneficial to the students in many ways, both in terms of the direct feedback they receive, but also because the process of thinking about, asking questions, and commenting on the work of other students can trigger reflection about their own work.

    What I’ve noticed running these workshops is that the students here are not as well-prepared for this exercise as the students I’ve taught at other universities. They simply don’t have as much experience in reading and commenting on other student’s work, and are not sure how to do it. Each semester there are also students who tell me that they think this is a waste of time because they want to focus on their own work. For the students to see the value of peer feedback (both to the student they are helping and to themselves), I believe we’ll have to work on changing some of the ‘culture’ of the university. Otherwise the students will just disregard or dismiss these opportunities.

    1. Hi Ellen, thanks so much for sharing how different your experience has been in European universities – I agree that this is something that is a cultural approach within a program or university that needs to be fostered from the first year right through the degree, as well as being explicit with students about the “why” and the value of peer feedback. We shared a bit about this in our previous coffee course on peer feedback – http://anuonline.weblogs.anu.edu.au/2018/06/05/day-2-getting-students-ready/ and http://anuonline.weblogs.anu.edu.au/2018/06/06/day-3-reliability-and-validity/ if that is of interest to you. We noticed that some schools/disciplines really prioritise and scaffold peer feedback, but others do not.

  10. I agree with Ellen’s comment, my immediate reaction to the first question was that it can be difficult to give effective feedback in the time we realistically have to devote to the task. I like the idea of using peer-feedback and self assessment to help make it more sustainable. I am also looking forward to learning more about audio and video feedback.

    The second reason I think that feedback in not always effective is because educators often keep the comments vague to limit challenges from students if they think the feedback doesn’t match the mark. However, it is nearly impossible for student to understand and learn from “vague” feedback. Perhaps clearly establishing with students to aims of feedback might help here.

    1. Indeed! Wouldn’t it be good to develop our own ‘vagueness detector’ to stop ourselves from making comments that will go over our students’ heads…?

      1. Hi Julia and Olga, another area where I have found “vagueness” in receiving feedback (thinking of when I was a PhD student) was that my supervisors often had trouble stepping back from their significant experience and expertise to explain it to me as a non-expert, and I often found their comments vague and difficult to action as a result! The curse of too much knowledge?

  11. The most common form of feedback, I believe, is exam mark due to multiple reasons brought up by comments above. Students will find it hard to relate to their situation to improve. Another important reason I believe that makes feedback less powerful is the misconception of some educators believing that feedback is to conclude the learning process, instead of part of the learning process itself.

  12. I give feedback in multiple ways: written feedback in addition to the grades of each perspective of the assignment; guide students to understand the weakness in their chain of knowledge after answering questions; self-paced online quiz and in-tutorial quiz.

  13. I actually found the feedback received recently for my paper is very constructive. The paper was not accepted for publication, but I found the referee report was very carefully written and helps me to re-tell my story and shift the research direction.

    1. Thank you for sharing your insights, Hua Deng. Sorry to hear that your paper was not accepted, but happy that the feedback that you got was useful. Hopefully, it’ll help you improve the paper and get it published! Also, thank you for your other comments re: exams and timing of the feedback.

  14. At one university that I taught at we used to ask all the students to submit a self-assessment along with a predicted grade with their work. They did not all rate themselves as HD and quite a few were very tough on themselves. The actual grades were often fairly close to the students predicted grade and it also helped them to reflect on their own performance rather than just wait to be judged by the lecturer.

    1. Thank you, Richard. That’s exactly what we’ll be recommending in our future posts! Self-reflection and assessment literacy are powerful tools in making feedback effective.

  15. I think learning to give feedback is a very useful professional skill as we all need to do this – in performance reviews, in work teams, as a technical expert and do on. Feedback is useful to the person receiving it and to the person receiving it. Outside of the course assessment setting, learning to receive feedback, value it and respond appropriately is important, as is learning to give feedback that is well received.

    Generating useful detailed feedback is hard because it requires detailed consideration of the submission (or behaviour) and how it could be improved. It is a useful development exercise for the assessor and I think we should maximise the opportunity by sharing practice – so where there are is more than one assessor – discuss and critique the feedback given. This is how we improve our own practice and develop expertise. So teaching staff should be involved in this development of feedback-giving skills, and students should also be given such opportunities.

  16. In some previous training sessions, I learned that powerful feedback should be personalized based on each individual’s behavior and how they accept the feedback. In this semester, I find feedback could be more powerful when it caters to the students’ needs.
    For example, there were two students who did very well in an assignment. One student might have a passion for this subject. For this student, useful feedback should provide some extensions of the course material or guide to the latest research. The other student only cared whether this was a HD or not (which is more common than I thought). Then useful feedback should give suggestions on how to develop a better argument.
    It takes some time to identify the students’ expectations for the course. But the current feedback from my students indicates that those comments work well. They take some time to think about the words. (I’m not going to argue whether only aiming for a HD is proper in the view of higher education…)

  17. Despite it being listed as a misconception, my experiences in giving and receiving feedback strongly suggest that there is a correlation between quality feedback that reaches its potential and time. Some of my worst (and sadly, most frequent) experiences of feedback came in the form of tick, tick, question mark, tick, grade, generic comment (eg, “Well done!”). This is not feedback, as it does not facilitate development and improvement. In contrast, quality feedback has only ever come in instances where the reader engaged with the work. The standard of quality depends on the level of engagement. When markers are expected to churn through 1,000 words in 20 minutes, they are not engaging with the material at any substantial level. Ask anyone who runs a research methods/methodologies course, and they always note the fact that it takes the average postgrad reader several hours to effectively engage with a standard journal article. Using technology might speed up the process of giving feedback, but it does not significantly affect the level of engagement.

    I am a native English speaker. I teach writing and research skills. I cannot engage with 3,000 words an hour. I am not ashamed of this fact. I happily spend more time than I am paid for to give quality feedback to my students. They love it. The primary feedback I receive from my students is that they wish that such constructive and useful feedback was an industry standard. This is reiterated in the improvement I see in my students across each semester, and throughout the rest of their degree.

    1. Hi Bhavani, I completely relate to your disappointment of getting “tick, tick, question mark, tick, grade, generic comment (eg, “Well done!”)” on numerous assignments. So I also knew that it would be important to devote ample time to providing constructive feedback when I started teaching. I wanted to assure my students that I’d read every word and given thought to all their thoughts. I was often surprised how many students were so grateful that I didn’t just skim their work but would actually take the time to read everything and write substantial comments as they had never had that before. I totally agree with you. Why would we set long writing tasks if we can’t be bothered to read or respond to the end products?

  18. I thought a lot of the comments above about timeliness around marking were definitely issues I experienced when I first started teaching. In order to combat that I prioritise what I think are the places where I can give the most assistance and where my value as a teacher sits. For example, I am not a spell checker or grammar police – if you are in a second year ANU course its not my job to highlight everytime you have a typo. Instead I make sure that students arguments are logical and their grasp of the more complex ideas that my courses are based on is tight. Ia also tend to pick up on lazy or cliched thinking or ask questions of things that are not wrong but could be viewed in another light to encourage them to think about issues more deeply. I aim to make this feedback feel like part of a continuing debate of ideas rather than a series of points deductions. I thought the comments above about formative feedback were particularly important – if we expect students to have a growth mindset that at any level they can and will improve them our feedback needs to facilitate that. As such I find the oddest part of my job is sometimes struggling to find some way exceptional students can improve in the future – I know there must be something but it does make you feel a bit like reviewer 2! (obviously worded much more positively though…)

  19. To share an example of bad feedback that seems to be very common in the hectic life we are living today is that paper/thesis reviewers have not had enough time to fully understand the claims/research questions and their entire feedback statement is killed by their incorrect assumptions of what was handed in by the student author. In this situation, explicit feedback eases rebuttal, but not many assessors admit their mistakes in an authentic way.

  20. I have changed my feedback style over the years. Generally I use a think aloud approach to give constructive feedback in text. I also provide 2-3 comments overall that focus on 1)what the students did well and 2)what they need to improve. I find that too many comments confuses students and they do not know what to focus on. Prioritising and focusing on the top 2-3 key areas allows students to set achievable goals to work on before the next assignment.
    I find that many students do not read the feedback on Turnitin. Turnitin has made it much easier to mark but the feedback is somewhat hidden for the students. They go to gradebook to see their mark and many do not do much more than that. Chris Browne has started to develop an alternative approach to providing feedback using a Wattle database. This generates a PDF linked to gradebook where students can get the mark and feedback in one document. I have found it provides greater impact on student learning and makes it much easier to moderate student work.

Leave a Reply

Your email address will not be published. Required fields are marked *

*