Written by Dr. Patrick Tran, UNSW Canberra
In this post we will cover briefly some basic concepts and provide pointers to further readings. We will not go into the technical details of learning analytics techniques and tools but rather provide you an overview of the topic.
What is Learning Analytics (LA)?
Learning Analytics (LA) is defined as the “measurement, collection, analysis, and reporting of data about learners and their contexts, for the purposes of understanding and optimizing learning and the environments in which it occurs.” (Siemens and Long 2011) Learners’ data include their digital footprints in learning environments, feedback data, demographics and enrolment details. LA aims to:
- induce relationships or patterns that are unsuspected otherwise and,
- summarize them in both understandable and useful ways to the stakeholders.
As described above, LA is not only about applying advanced data science techniques on the vast amount of data to afford insights into learning, but also interpretation of these findings. LA can help group learners based on their learning behaviors (clustering), identify outlier records that may indicate poor performance, and induce potential correlation between learners’ engagement and their academic progression.
LA is a multidisciplinary field that intertwines instructional design, psychology, statistics and knowledge discovery to develop a unified explanation of human learning. There are a number of research fields with similar objectives as that of LA, including:
- Academic Analytics: applies statistical techniques on large academic data sets to inform pedagogical practices and institution-wide policies. To many LA practitioners, the difference between Academic Analytics and Learning Analytics is more terminological than conceptual.
- Educational Data Mining (EDM): applies computerized methods to search for patterns in huge educational data sets. EDM is said to transform “unintelligible, raw educational data” into actionable intelligence that is “legible and usable to students as feedback, to professors as assessment, or to universities for strategies” (Romero et al 2016). Unlike Academic Analytics that takes a holistic approach to analyzing the learning experience as a whole, EDM implements a “reductionist approach” by focusing on the reduction of large volume of data to smaller, more manageable components and discovering relationships among them (Siemens and Baker 2012).
For the sake of simplicity, we will refer to LA as both Academic Analytics and EDM in this course.
A Typical LA Workflow
A large number of LA frameworks can be found by reviewing the relevant literature. Most of them fit into the high-level generic process or workflow below, that I often use in my analytical work.
This framework collects and processes data related to learning, and interprets and feeds the analytical findings back to the previous steps. The framework components are described below:
- Analytic Questions: first, we define the analytic problems or objectives by formulating specific questions for investigation, e.g. “measure student engagement in online forums”, or “what is the relationship between student participation in online forums and course outcomes?”.
- Pedagogical Insights: identify learning theories, assumptions or observations that may be relevant to this LA application. These discipline-specific insights help guide the other LA processes (Data Processing and Interpretation) and narrow down the search space of solutions to the underlying questions. For example, literature may suggest that the vocabularies used in form discussion may indicate the quality of students’ engagement and hence the collection of this data for our LA project.
- Data Collection: retrieve data from sources.
- Data Processing: organize data and transform it to meaningful and useful knowledge. I used MS Excel, Python, R and NVIVO for this purpose, depending on the data types and contexts.
- Data Preparation: pre-process raw data for further analysis. This includes data cleaning (missing data, errors), data filtering, feature engineering (derive new variables calculated from existing variables).
- Analytic Engine: apply analysis methods on the prepared data. Depending on the type of data and analysis objectives, different knowledge-discovery methods can be implemented, including quantitative analysis (statistical – descriptive and inferential, Data Mining) or qualitative analysis (thematic). Data visualization can be used to assist these methods in validating or illustrating the analysis results.
- Data Interpretation: discuss the analysis results from the previous step. Where appropriate, the results should be contrasted or compared against prior experiences, synthesized and generalized. The educational context of the LA application should be considered when interpreting the results. The analytic outcomes generated from this step could be intelligent feedback to students for an assessment, learning adaptation and personalization, and new knowledge that is useful and actionable.
- Feedback: provide feedback to the previous steps. In this step, we:
- Address the analytic questions.
- Deliver the interpreted analysis results to the intended recipients, i.e. the stakeholders. This is when you write up a report describing your findings, action plans and recommendations. In some user-facing LA applications, a dashboard is used to deliver the insights back to the users.
- Modify the previous steps based on the insights discovered, e.g. changing parameters used in the analytic engine for better results, or validating the pedagogical insights used.
- LA Stakeholders: LA aims not only to assist (directly) learners and educators, but also (indirectly) other stakeholders.
Educational data contains students’ digital footprints in virtual learning environments. In particular, it is any data on student learning and academic performance, demographic details, and course-specific data (activities, resources). This rich data is scattered around multiple systems, including learning management systems (LMS) such as Wattle (Moodle) and student information systems. In addition to data generated and stored in virtual environments, real-world contextual information such as classroom settings and face-to-face interactions can also be used to ensure a complete understanding of the learning experience.
For a more detailed overview of educational data types and sources, please check out the longer version of this post.
Why adopt LA in your teaching and learning?
The main objective of LA is to enhance student experience through evidence-based analytical intelligence and evaluation. Drivers for the adoption of LA include its abilities to:
- Explore opportunities to leverage information about learners from multiple sources and various data types, ranging from extracurricular activities, social media postings to demographics details (age, gender), physiological metrics and face-to-face behaviours captured by activity trackers and surveillance cameras. This rich data enables LA to infer learners’ intention, cognitive and metacognitive processes or even stress level during learning activities (Zeide 2017).
- Analyse large volume of learning data available in online platforms such as A massive open online courses (MOOCs) and social media networks.
- Discover new patterns and actionable knowledge about learning processes that might otherwise go unnoticed. The insights generated by LA can help drive learning improvements and inform institution’s strategies.
- Meet the diverse needs of a wide range of stakeholders interested in what LA can do, including learners, educators, researchers, institutions and government agencies. The insights generated by LA are also delivered to end users in many forms such as periodic intelligence reports, ad-hoc reports, real-time dashboards embedded in a web page or a mobile app.
For more examples on LA, or an overview of commonly used LA techniques, please check out the longer version of this post.
In your opinion, who are the stakeholders in an LA application, i.e. who can affect or be affected by the application? What are their interests and skills/knowledge required (if any)?
Share your thoughts by posting a comment in the discussion forum.
C. Romero, Cerezo, R. , Bogarín, A. and Sánchez‐Santillán, M. , “Educational process mining: a tutorial and case study using Moodle data sets,” Data mining and learning analytics: Applications in educational research, S. ElAtia, Ipperciel, D. and Zaïane, O. R., ed., pp. 1-28: John Wiley & Sons, 2016.
G. Siemens, and P. Long, “Penetrating the Fog: Analytics in Learning and Education,” EDUCAUSE Review, vol. 46, no. 5, 2011.
G. Siemens, and R. S. J. d. Baker, “Learning Analytics and Educational Data Mining: Towards Communication and Collaboration,” in Proceedings of the 2Nd International Conference on Learning Analytics and Knowledge, Vancouver, British Columbia, Canada, 2012, pp. 252–254.
E. Zeide, “The limits of educational purpose limitations,” University of Miami Law Review, vol. 71, 2017.