Written by Dr. Patrick Tran, UNSW Canberra
There are many LA applications reported in the literature. We will explore some simple LA tools available in Moodle (Wattle for ANU users) that should be accessible to teaching staff. Your own institution’s LMS will likely have similar tools available.
For more diverse LA examples, please check out the longer version of this post.
LA Example 1: Monitoring Students’ Online Behaviors With Course Reports
As learning data is stored in various systems, retrieving and combining them in a meaningful way can be challenging. In Moodle, instructors can gain insights into student engagement with their courses and communicate with certain students. Most of the built-in reporting capabilities in Moodle are centered around the user access log, which contains user ID, course ID, action, event, target, timestamp and origin of the access entry. Based on this log data, a number of reports can be created to support different user views. Just go to the Settings / Reports menu to explore the reports. (For users at ANU, this is available under the Settings wheel -> More -> Reports).
This data may be useful to monitor students’ online behaviors, e.g. how often and when a course resource is viewed or modified by students. For example, you can check if a particular student has accessed a particular item in your Moodle site.
This report displays the number of views by activities (forum, page, assignment dropbox …) and resources (files). This report can give you a sense of how particular activities and resources are being used.
If you enable the “Completion tracking” option under Course settings and configure “Activity completion” under Settings of an activity, you will find the Activity Completion report under Reports menu.
This creates a customised report that you can use to track whether students have completed the items of the course according to the criteria that you identify.
This report displays access statistics of a course over a period of time. This report can be filtered by role, by action (views/posts). I use this report to understand the patterns of user access to the course during the semester.
This report shows students’ participation for a particular activity measured by the number of actions (view or post) during a set time-frame. There is an option to send a message to some selected students. I use this report all the time to identify and follow up with those students who did not engage well with the course.
All data presented in above reports are summarized in the following diagram:
For a detailed description of these reports, please refer to the Course Reports Moodle Documentation.
LA Example 2: Quiz Data
Scenario: Online quizzes have been widely used in eLearning as both a formative and summative assessment tool. Instructors can utilize the auto-grade, instant feedback and randomization features in quiz platforms such as Moodle Quiz. We can in fact collect much more data about the assessment process with these online quizzes than paper-based tests. With some simple data processing and visualization steps, educators could at least gain some good understanding about how their students performed in a test (scores achieved, time spent) or indicative levels of difficulty of the questions used. I often conduct an Item Analysis to measure the psychometric quality of the questions in my major assessment items. For more details visit Item Analysis.
About the data and technology used: Quiz data is collected from the built-in “Results” reports found in the quiz settings page. This includes students’ grades, actual responses and time spent. We can also link student’s demographic information to this performance data if necessary. You then graph this data in MS Excel. I however prefer Tableau as I like its fast and intuitive drag-and-drop approach to visualizing data.
About the LA application: Below is the findings presented as a “data story” in Tableau for an online course that involved two cohorts of students and three tests in Moodle. From the graph below, we found that students performed very poorly in Test 3, question 5 across all cohorts. Further investigation may help confirm that the question was in fact faulty or the knowledge tested by that question was not covered enough in class.
By analysing your quiz results, you can feed-forward into the rest of the course or future offerings of the course to address any issues in students struggling in particular areas. This information can help you as a teacher adjust your approach in the future to support students in relation to this very specific area of need.
LA Example 3: Open Text Data
Scenario: Student feedback data can be collected in many ways, using interviews, surveys or suggestion boxes. This data more often than not contains open text data. We can conduct a thematic analysis of this rich data to identify the common themes.
About the data: Text from the transcript of student interviews.
Technology used: I used NVIVO to analyse the text data while visualizing my findings by Tableau and wordart.com for word clouds.
About the LA application: A word cloud is first created to highlight the issues the respondents talked about most. For each identified theme, the number of respondents who mentioned the good aspects (what worked) and things that need improvement (what didn’t work) are counted. This data is visualized in a dual-axis bar chart.
Other LA Tools
In addition to the built-in analytic reports, Moodle has a large repository of plugins related to LA. Check out the Moodle Documentation for a full list of LA tools.
Another sophisticated LA system which helps instructors personalize engagement with students at scale is the Student Relationship Engagement System (SRES). SRES has been piloted and used in a number of major universities such as UNSW, University of Melbourne, and University of Sydney. SRES can be used for attendance checking, sending personalized emails based on if-then rules, personalized feedback to students. Visit SRES for further information.
Please share your experience with LA in your capacity as either a student, instructor or administrator.
If you know or have used an LA tool in your teaching and learning, please share:
(1) a brief use case,
(2) any detail on data and analytic techniques; and
(3) how the analytic results are delivered to you (e.g. report, dashboard or any user interface).
Post your response to this activity in the discussion forum.