Written by Dr. Patrick Tran, UNSW Canberra
Along with the promise and potential embodied in LA come mounting challenges including ethical issues and the pitfalls related to data interpretation. We will explore these issues and their potential solutions in this post.
Educational institutions have become more concerned about the privacy and security of their students as a result of the mass collection and centralization of student data, and the recent data breach incidents (including at ANU). The LA ethical and privacy issues can be summarized into 7 categories: (1) Privacy; (2) Informed consent, transparency and de-identification of data; (3) Location and interpretation of data; (4) Management, classification, and storage of data; (5) Data ownership; (6) Possibility of error; and (7) Role of knowing and the obligation to act (Steiner et al., 2011).
Several works have been conducted to propose a systemic approach to addressing ethical concerns of LA, including the “Code of Practice for Learning Analytics” (Sclater, 2016) and the “OECD Privacy Framework” developed by Organization of Economic Cooperation and Development (OECD, 2019).
The privacy taxonomy introduced by Solove (2006) provides a comprehensive overview of potential harms related to wrongful treatment of personal information during the data analytics process.
From this framework, we could infer what impacts LA can have on learners if not used properly. For example, assessment analytics helps learners track their own progress but may also cause them performance-related stress. As reported by Mayer-Schönberger and Cukier (2014), misuse of past performance data could have grave consequences, e.g. dismissing students’ ability to change and forcing them out of higher education. Similarly, constant fear of surveillance can adversely impact learners’ psychological well being. Furthermore, collection and correlation of learners’ data at an unprecedented scale regardless of need makes the learners vulnerable to identity theft and other privacy breaches. These are real risks and it is negligent for anyone to disregard them.
To address the privacy concerns, many universities have implemented strict privacy policies, data management and human research ethics procedures. These measures attempt to define ownership and stewardship of LA data, duty of care in data management as well as ethical requirements for the use, analysis and reporting of LA results. The process and techniques used in LA applications must be transparent, to the highest possible degree, to everyone involved (Beattie et al 2014). This includes what data is being collected, how it is being aggregated, what benefits and risks it may have, and who it is being shared with.
Data Interpretation and Intervention
Reaching valid conclusions after analyzing data is a critical step in the LA process as it has detrimental effect on instructional decision making. Overgeneralization of learning data may be caused by a number of factors, including biased data or analytic techniques. This can disadvantage certain learners by, for example, imposing coarse indicators of academic success on the entire learner population that undermine individuality instead of jointly defining with individual students what success really mean for themselves (Dishon, 2017).
“Pedagogic lurking” is reported as a widespread engagement phenomenon in online courses, with some perceiving it as problematic and others as a step towards more active participation. In particular, learners who do not participate in an online learning environment in an active way are referred to as “lurkers”. It is important to make clear what “active participation” means. We often measure participation and engagement through visible indicators such as the number of messages, their word count and word choices learners use in an online discussion forum. However, as pointed out by many researchers (Honeychurch et al., 2017; Dennen, 2008), these indicators may not show the entire picture of learner engagement. Some learners may be cognitively engaged in active information processing and motivated to learn without showing any visible signs. As a result, we should include other avenues into our attempts to measure learners’ engagement. These could be self-report questionnaires and in-class observations. For a further discussion of this, check out our previous course on Student Engagement.
In a well-designed LA application, data and findings should be summarized in a dependable and accurate manner by not just interpreting it through a qualitative and quantitative lens, but also through consideration of many relevant factors and stakeholders. Beyond describing what obvious on the graphs and tables or counting occurrences, the analysts should make sure appropriate measures are used and compare the observed findings with other cases. In many cases, education domain knowledge such as learning theories and course-specific context can be used to guide this interpretation process. Finally, conclusions should be drawn to answer the research questions if any. Where appropriate, these conclusions should be synthesized and generalized such that the discovered knowledge is actionable and can be applied in new or similar contexts.
Finally, changes and remedial actions resulted from LA findings must be designed and planned with strong pedagogical ground, careful consultation with both the educators and learners. If a change is institutionalized without a proper learning context, a teacher could feel pressured to make arbitrary pedagogical decision, such as forcing students to post in online forums even though discussion had already taken place in the classroom.
LA opens doors to great opportunities for educators and learners to afford insight into how learning takes place. Analytic outcomes and recommendations resulted from learning data promise enhanced learning experience, but not without precautions against misuse and misinterpretation of the data. Some people naively believe that “some analytics is better than nothing” but this is not always true, LA can also be worse than nothing if it systematically ignores important indicators other than the available data and leads to harmful changes to the learning environment. Because there are so many ways in which learning data can be misinterpreted, the line between the promise of “personalization” and the danger of “discrimination by design” is fine and blurred.
Similarly, the obsession with more data and more analysis poses mounting challenges in protecting data and safeguarding its use. Finally, the need to include an ethical dimension in the applications of LA has become urgent and important due to significant concerns in data privacy and security.
Choose any or all of the following questions and post your responses in the discussion forum:
- Will you use LA to support your teaching, and how? Assume that you have access to the tools and support you need.
- In your opinion, what are ingredients of a successful LA application?
- How would you measure students’ engagement in a blended course that contains a portion of face-to-face instruction and some online learning activities? Give an example of what conclusion can be drawn from your measures.
We would love your feedback!
Our team is currently running an evaluation of the Coffee Courses, and we would really appreciate if you could take our survey and share your experiences.
Beattie, S, and Woodley, C, and Souter, K (2014) Creepy Analytics and Learner Data Rights. In: Australasian Society for Computers in Learning and Tertiary Education (ascilite2014), 23 – 26 November 2015, Dunedin, New Zealand. https://research.moodle.org/84/
V. P. Dennen, “Pedagogical lurking: Student engagement in non-posting discussion behavior,” Computers in Human Behavior, vol. 24, no. 4, pp. 1624-1633, 2008. https://www.sciencedirect.com/science/article/pii/S074756320700115X
G. Dishon, “New data, old tensions: Big data, personalized learning, and the challenges of progressive education,” Theory and Research in Education, vol. 15, no. 3, pp. 272–289, 2017. https://journals.sagepub.com/doi/full/10.1177/1477878517735233
S. Honeychurch, A. Bozkurt, L. Singh, and A. Koutropoulos, “Learners on the Periphery: Lurkers as Invisible Learners,” European Journal of Open Distance and E-Learning, vol. 20, no. 1, pp. 192-212, 2017. https://www.eurodl.org/?p=current&sp=full&article=752
OECD. “The OECD Privacy Framework,” 2019; http://oecd.org/sti/ieconomy/oecd_privacy_framework.pdf.
V. Mayer-Schönberger, and K. Cukier, Learning with big data: The future of education: Houghton Mifflin Harcourt., 2014.
N. Sclater, “Developing a Code of Practice for Learning Analytics,” Journal of Learning Analytics, vol. 3, no. 1, pp. 16-42, 2016. https://epress.lib.uts.edu.au/journals/index.php/JLA/article/view/4512
D. J. Solove, “A taxonomy of privacy,” University of Pennsylvania Law Review, vol. 154, pp. 477, 2006. https://scholarship.law.gwu.edu/cgi/viewcontent.cgi?article=2074&context=faculty_publications
C. M. Steiner, M. D. Kickmeier-Rust, and D. Albert, “LEA in Private: A Privacy and Data Protection Framework for a Learning Analytics Toolbox,” Journal of Learning Analytics, vol. 3, no. 1, pp. 66–90, 2016. https://epress.lib.uts.edu.au/journals/index.php/JLA/article/view/4588