Assessment and Feedback

Day 3: Building successful ePortfolio activities

Building successful ePortfolio activities

Jenny Edwards and Aliya Steed, ANU Online and Shane Nuessler, Manager, Learning Information and Environments, University of Canberra

Image source: Pixabay.com retrieved 22/8/17

Any tool is only as good as its implementation. We’ve probably all heard of (or been guilty of) cases where a discussion forum has been just flung into an online course, and then we wondered why the students didn’t magically fall into a vibrant discussion. This is even more the case for ePortfolios. The tool is designed to enable online portfolio learning, with goals as discussed on Monday, including deep engagement, reflection and metacognition. While a portfolio can revolutionise a course or program, successful implementation is not so much about the availability of the tool, but about the extent to which portfolio learning is adequately integrated, supported and the new approach to learning is valued by both teachers and students (van Tartwijk and Driessen in Trevitt, Macduff and Steed 2014: 75).

So an important task is to think through how any proposed portfolio activity is designed and embedded in its wider context. Experience indicates an ePortfolio activity is more likely to be successful if it is:

  • Well-integrated – fits in clearly as part of the course or program, not just an optional added extra
  • Well-motivated – directly relevant for students, clearly understood benefits for learning, appropriately linked to assessment
  • Straightforward for the students to use and adequately supported.

In this post, we will explore some of the challenges to using portfolios in your course or program, and look at some of the things that teachers can do to help a portfolio work effectively.

Challenges

Portfolio learning is often a new and challenging way to work for students. Many of the activities that are essential to portfolio assignments can be confusing to students. Drawing on the literature and the discussions over Day 1 and Day 2, here are some of the key challenges in implementing a portfolio.

Students feeling confused or uncertain  

Image source

Portfolios represent a significant shift in learning and assessment which can result in student resistance, especially without considerable guidance and support (e.g. guidelines, examples) throughout the process (Butler 2006: 4).  The scope and purpose of portfolios is often drastically different from traditional assessments, which can cause anxiety and concern among students. As pointed out by Paul Francis in his comment replying to Monday’s post, students can lack the very skills they need in order to successfully build reflective and metacognitive skills.

 

Cognitive load of learning a new platform

As well as understanding the nature of portfolios, both students and staff may face the additional challenge of learning to use what may be a second or third learning technology platform. Support can help, as well as a focus on using systems which are easy to use and as “transparent” as possible. 

Concerns about privacy and exposure

While it can be a valuable learning experience for a student to reflect on things that have perhaps not gone well or that demonstrate their weaknesses, this also makes them vulnerable.  Stiofán Mac Suibhne (2016) tells the story of a student studying to be an osteopath, who realised upon reflection that he was very poor with knees.  As a result, he went on to study knees more carefully.  He became a world leader in the field.  However, a quick Google search of his background still delivers his “I’m hopeless with knees” reflection, which could potentially undermine the confidence of potential clients. Students need to feel that they will be protected with appropriate confidentiality.  The privacy of clients or patients also needs to be protected, such as in placements or work-integrated learning situations.

Image source

Ownership and portability

If we expect students to invest in their portfolio as a personally significant piece of work, can they expect to be able to retain access when they graduate? Who retains the intellectual property?  Students need to be clear about these issues upfront.

Assessing portfolios can be difficult.

Traditional assessment notions of “validity” and “reliability” may not be compatible with the personal and formative nature of portfolios (Trevitt, Macduff and Steed 201: 71).  As Butler points out, “the danger is that learning and reflection will get lost in the drive to measure competency” (2006: 4).  In designing portfolio assessment, teachers find themselves navigating the vast range of possibilities as to what constitutes evidence of learning (e.g. images, video, interactive 3D maps, music and so forth) and balancing the level of prescription and guidance. Too much can lead to resentment, too little to defensiveness and superficiality (Butler 2006: 4).

Further issues both the teacher and the institution may need to deal with include:

  • accountability, academic integrity or identity misrepresentation
  • social and professional identities, where the nature of portfolios has blurred with social media tools
  • the perception of increased workload (Trevitt, Macduff and Steed 2014, 71) 
  • students leaving it too late
  • integration of portfolios over multiple courses or entire degree programs

 … and many more!

Activity:

What concerns you about using an ePortfolio in your teaching practice? What do you think will be the key challenges for your course when implementing a portfolio? Post your answers in the forum.

Success factors 

As well as identifying the challenges teachers may face in using ePortfolios, there are also factors which can increase the chances of success.  Butler (2006, 4) provides us with a summary of criteria for success:

  • Familiarity with the portfolio concept, including an understanding of both the process and the product of portfolio construction;
  • Clear framework and guidelines;
  • Structure tempered with freedom for creativity;
  • Feedback during the evidence collection process;
  • Understanding the value of reflection;
  • Understanding of the value of the portfolio for future use such as employment;
  • Motivation to learn and achieve good marks;
  • Student ownership of the portfolio;
  • Making connections between the portfolio content and the outside life of the student;
  • Consideration of the target audience; and a
  • Sense of achievement at overcoming initial struggles to understand the portfolio concept.

How much each of these criteria might be emphasised will vary with the purpose of the portfolio, the learning goals in mind, as well as stage and nature of the student group.

Where to from here?  

Given the wide variety of possibilities which ePortfolios afford and the potential sensitivities and complexities that universities face in deciding how and why to implement them, an institutional-level conversation is probably a necessity, rather than a nice-to-have.  If you are considering experimenting with ePortfolios in your teaching it may worthwhile connecting with others already using them, not only to benefit from their experience, but in order to help shape the way ePortfolios are used within the institution as a whole. 

Upcoming Mahara ePortfolio training at ANU

Staff located at ANU are welcome to contact ANU Online to discuss their ideas for using portfolios.  They may also like attend one of the upcoming sessions which will demonstrate how the ANU portfolio system, Mahara, can be used. Please sign up for the next session, Monday 11 September from 1pm – 2:30pm in the Hancock Library Flex Studio.

Activity:

List the current assessment methods you use; list as many strengths and then limitations as possible. Looking at the limitations, is there a valuable range where portfolio practice can fit in?

The lists of challenges and success factors above are not exhaustive.  What design considerations would be important for an ePortfolio activity in your course or program? 

Further reading:

References

  • Butler, P. (2006) A Review Of The Literature On Portfolios And Electronic Portfolios (eCDF ePortfolio Project). October. Palmerston North, New Zealand: Massey University College of Education (Retrieved 5 August 2017, from: https://akoaotearoa.ac.nz/download/ng/file/group-996/n2620-eportfolio-research-report.pdf)
  • JISC (2008) – Effective Practice with ePortfolios – Supporting 21st century learning (Retrieved 22 August 2017, from https://www.jisc.ac.uk/media/documents/publications/effectivepracticeeportfolios.pdf)
  • Mac Suibhne, S. (2016). Panel plenary session ‘Give an example of how collecting evidence and connecting it to criteria can demonstrate your professionalism/leadership’ presented at 2016 Eportfolio Forum, ‘Connecting learning to the future’, meeting of ePortfolios Australia, Sydney Conservatorium of Music. September. Sydney. Australia. See program available at https://eportfoliosaustralia.wordpress.com/forums/2016-eportfolio-forum-home-page/, Retrieved 21 August 2017.
  • Pachler and Daly (2011) Key issues in e-Learning: Research and Practice.  London: Continuum.
  • Trevitt, C., Macduff, A and Steed, A. (2014) “[e]portfolios for learning and as evidence of achievement: scoping the academic practice development agenda ahead” Internet and Higher Education 20, 69-78.

12 thoughts on “Day 3: Building successful ePortfolio activities

  1. Hello
    I touched on this in my last post:
    – Students may need guidance on aesthetics – maybe teachers can utilise the design school for guidance?
    – It needs to be valuable – as mentioned in this post, there’s no point in doing an ePortfolio unless it’s going to add value. It’s easy to put new technologies in place however sometimes it’s best to keep things simple. I would see an ePortfolio a good thing if it opens up for the whole class – or maybe group assignments, though you wouldn’t want this to discourage people meeting face to face

  2. * What concerns you about using an ePortfolio in your teaching practice?

    I need to make clear what are the assumed skills, what I am teaching and what I am assessing.

    STEM students may not have the level of literacy, and in particular the experience in reflective writing, needed to complete an e-portfolio. If those skills are not listed as assumed, or required, for a course, then the course has to teach them, as part of the e-portfolio exercise.

    An e-portfolio might be used just as a tool, to help the learning and assessment. However, if the ability to produce an e-portfolio is being assessed, then that needs to be made clear, before the course starts.

    The e-portfolio exercise may require the student to reveal information about themselves, their acquaintances and their employer. The extent to which that is required, and the protection of the information provided, needs to be made clear to the student before the course commences. In the most extreme case, commercial or national security could be compromised, or the life of the student placed at risk, by a e-portfolio exercise.

    * What do you think will be the key challenges for your course when implementing a portfolio?

    The major challenge I found is to get the student to start on the e-portfolio early in the course. The tendency of any student is to leave an “assignment” until shortly before the deadline. Simply telling the student that an e-portfolio is different and they need to start early does not work. The solution is to scaffold the work, with small exercises with deadlines and assessment, throughout the course. The student is then prompted to work on their e-portfolio progressively.

    I have direct experience of how students don’t do e-portfolios. I spent three years as a graduate student, ignoring the reminders from my instructors about starting my e-portfolio for the first two and three quarter years. It was only when enrolled in the final capstone course, and a concurrent course on e-portfolio pedagogy, that I started. I then regretted not paying attention earlier.

    ps: Arthur: “… I really wish I’d listened to what my mother told me when I was young.” Ford: “Why, what did she tell you?” Arthur: I don’t know, I didn’t listen.” From: The Hitchhiker’s Guide to the Galaxy, by Douglas Adams 😉

    1. Thanks for your thoughtful input, as ever, Tom!

      I think you’re exactly right about clarifying upfront which skills are presupposed and which are intended to be developed throughout the the course. If (e.g.) reflective practice is an intended outcome of the course, there needs to be scope within the course for it to be learned. If there is not time/space/coaching for its development among the content course, it is probably not “fair” to assess it — for all that we may think it is a good thing.

      I was surprised a few months ago to hear one of out CBE lecturers was giving students 10 min at the end of each tutorial to write their ePortfolio, as it seems initially to be rather a prescriptive approach for a university course. However, as the course very explicitly included learning about, and learning to use, metacognitive techniques and skills, investing the class time like that communicates the value of activity, is well ‘constructively aligned’, and certainly gets the students past the problem of leaving it all to the last minute.

  3. Thanks Asha, that’s some useful thinking. One of the QUT best practice strategies is “ePortfolio pedagogy must drive the technology and not the other way around” (see our Further Readings). If we work backwards from our intended learning outcomes, through learning experiences to the activities we design, also bearing in mind that all our tools have pros and cons, we seem to find places where the ePortfolio gives a valuable combination of features (affordances) not readily available elsewhere.

    Yes, ePortfolio, as it is implemented at ANU, could be used for a whole-class assignment, certainly for group collaboration (although, your grading processes may lead you to look at other tools as well, e.g. OU Wiki).

    Re your comments here and on yesterday’s post about aesthetic guidance, a couple of thoughts spring to mind:
    – Many of our academic disciplines have the effect of minimising students’ opportunities to be involved in aesthetic design (e.g. see Tom’s comments yesterday about using styles). Some students appear to delight in the visual and creative opportunities of the ePortfolio (similar to Rebecca’s comment about converting her essay to an ePortfolio page), resulting in a more personal and deeper interaction with the issues and content for those students. Other students will feel defensive – “graphics is not my thing” — so I guess we need to design to accommodate both ends of this reaction as well as the spectrum in between.
    – If the aesthetic component is to be graded, then certainly there should be some guidance provided, exactly as for the graded content components.
    – Within ePortfolio at ANU, we could fairly easily set up a few different template pages that the students could take a copy of, implementing a handful of different aesthetic designs. However, similar to Butler’s “Structure tempered with freedom for creativity” comment, a number of teachers experienced with using ePortfolios have pointed out that, while students love to be given examples, providing an example risks collapsing the students’ own creativity, as they just replicate the example. So the answer may be to provide several examples, including commentary about where each has strengths and weaknesses (although given the personal nature of portfolios, these can be hard to find), or scaffold the students into the process a different way.

  4. Asha, I suggest e-portfolios can add relevance and flexibility to a program.

    As an example, Henry (2017) concludes that Australian Master of Cyber Security programs do not meet the requirements of industry. They suggest “… mission-specific and purpose-driven courses may better prepare students and address the skills crisis than generalist degrees.” (p. 3).

    I suggest e-portfolios might provide a framework for MCS , into which specific industry courses and certifications could be inserted, as required. There will be a seminar about the report, 11am, 24 August, at UNSW Canberra.

    References

    Henry, A.P. (2017, August). Mastering the Cyber Security Skills Crisis: Realigning Educational Outcomes to Industry Requirements, Australian Centre for Cyber Security (ACCCS), UNSW.

  5. What concerns you about using an ePortfolio in your teaching practice? What do you think will be the key challenges for your course when implementing a portfolio?
    I’d be concerned with the idea of ownership – if you can’t easily ‘take’ the ePortfolio with you, then it might be hard to get students on board. In addition, if there is a steep learning curve with using the platform, or if students feel it’s not giving them anything useful, this could be a challenge to implement.

    List the current assessment methods you use; list as many strengths and then limitations as possible. Looking at the limitations, is there a valuable range where portfolio practice can fit in?
    – tests, quizzes, assignments, exams, essays, short-answer questions, multiple-choice, lab books, oral presentations, critiquing journal articles…
    The strengths and weaknesses mostly revolve around whether the method correctly aligns with the desired learning outcome so I can’t really think of any limitations that don’t reflect a problem in alignment. However, I see ePortfolios as being a good forum for students to articulate essays, articles, maybe lab books?, while keeping in mind a good overview of what they are doing any why. I think this, maybe because the ePortfolios seem to allow you to see the work more holistically (e.g., compared to writing a paper with a beginning, middle, and end)?

  6. What concerns you about using an ePortfolio in your teaching practice? What do you think will be the key challenges for your course when implementing a portfolio?

    Most of our students are new to using an ePortfolio so my biggest concern is ensuring there is space within the teaching schedule to allow time to familiarise students with Mahara, the purpose and alignment to learning outcomes and to provide examples.

    I think another key challenge would be to ensure consistency in the use of ePortfolio across all units within the curriculum. I feel that this would be important to allow a smooth introduction to students. If all the basic elements/templates are the same across units they may find it easier learn how to use it and to then adapt their portfolio to provide an authentic and personal representation of their learning.

    List the current assessment methods you use; list as many strengths and then limitations as possible. Looking at the limitations, is there a valuable range where portfolio practice can fit in?

    Assessment methods – exams (summative), interactive quizzes, review questions, peer review (formative)
    The limitations for assessment in my unit relate to current curriculum requirements which are governed by our professional registration body, which are at this point not subject to change. It can be very difficult for students to demonstrate clinical application of conceptual knowledge using just exams. I think a portfolio would be of immense value here, allowing for reflection and peer review. I believe it would also allow students to more clearly see the links between some of the more theoretical and practical units.

    What design considerations would be important for an ePortfolio activity in your course or program?

  7. I’ll echo some of the concerns others have raised regarding the time it takes to learn and get students used to eportfolios. Marking also is a concern. I have a large reflective assignment where I am relatively flexible in terms of what students write about and how they do so. But I had many concerns raised by students who wanted an exact structure for how they presented their learning. This made it difficult because in providing the students with the exact structure that they desired, almost every student ended up reflecting in the same manner (which we know they don’t in reality).

    The methods I currently use are similar to Angela and I agree with her that it is important to ensure that the method aligns with the learning outcome.

    1. This is a great tension that you have pointed out David – students often seek direct instructions and structure for assignments that they can work towards when something more expansive like the large reflective assignment you mention can benefit from being less explicitly organised and allowing some more options in terms of how it is presented. I think this is a key issue with types of “non-traditional” assessments, and highlights the value of learning outcomes matching assessment methods, as you say.

  8. I am most concerned with privacy and exposure. As can be seen from Day 2 material, a feature for this ePortfolios technology is “reflective”. First, I think few people would like to expose their weakness to the public. Besides, it’s easy to be over-emotional when writing such reflective articles. Those emotions will, on the other way, affect the tone of the articles. Those two negative factors will limit the contents of the ePortfolio. Meanwhile, it would be difficult to maintain the consistency in marking.

  9. Hi Sunny, your comment reminded me of an article I read where a doctor’s journal may have been used against her – she was charged with manslaughter due to apparent negligence. Privacy and exposure are certainly important considerations when using ePortfolio and encouraging reflective practice in general as we can be very vulnerable in a number of ways engaging in these activities. Here’s the section of the article I mentioned:
    ” After Jack died, Bawa-Garba wrote a “personal reflection” – a vital part of modern medical training. When Bawa-Garba was charged in December 2014 with “negligence manslaughter” over Jack’s death, it was reported that this reflection was used against her in court. That claim is subject to some dispute but has caused alarm in the medical world, because any move to use a doctor’s journal in this way could adversely affect medical teaching worldwide.”
    https://www.smh.com.au/opinion/death-of-british-boy-has-worried-junior-doctors-all-over-the-world–with-good-reason-20180201-h0rxh5.html

  10. Concerns /key challenges:
    At a broader level, my main concern as a teacher (vs as a PhD student, which I have already covered in Day 1 post), is the prescriptive nature of an eportfolio, where the categories/examples/templates we set for inclusion within portfolios, takes on the normative value of ‘these items are considered worthy academic outputs’, to the exclusion of others. So treading very carefully around what is included. This can redefine our conceptual understanding of what research outputs look like, which is pretty heady stuff. An example, considering ‘visualising a thesis’ as something that a HASS researcher needs to do – how do you ‘sketch’ the complex dynamics of multilateral negotiations? A smiley face with a megaphone? On key challenges, making sure that we are not overloading our students assessment load and setting them up to fail. We have to be very wary of adding in yet another thing for students to do. We need to always bear in mind the workload, and ‘brain’ load, of our students. (Previous posts will have already flagged that my default setting vis a vis students is that they are hear to learn, hard workers, and worthy of our respect unless proven otherwise, so absolutely we should not be pushing them too hard).

Leave a Reply to Tom Worthington Cancel reply

Your email address will not be published. Required fields are marked *

*