Data is the Foundation for Progress

Nearly all institutions today are using Learning Management Systems (LMS) and Student Information Systems (SIS), which provide us with endless sources of information on student and faculty behaviors. This data can then be mined for clues on in-course retention, program persistence, quality of student learning, and admission demographics correlated to student success.

The emerging field of academic analytics is applying the same types of pattern-based strategies to education that Amazon.com has used in the retail industry to benefit consumers. Amazon.com mines data on viewed content, purchase history, and wish lists and pairs that with demographic data to recommend products that similar users have purchased when looking at a product of interest. For example, I am considering the purchase of a navigation system for my car. When I typed navigation system into the search field it gave me a list of accessories that other users have purchased when buying the product I clicked on. While there is a clear profit motive, I would indeed need to buy an accessory to mount the product to my car which is helpful information for me to consider before making a purchase decision.

In a similar fashion, data scientists can mine LMS and SIS systems for information on student performance (both grades and learning outcome scores), activity by feature or by content object to come up with actionable at-risk dashboards for academic leaders. Figure 1 illustrates an at-risk student dashboard in Pearson eCollege’s Enterprise Reporting tool. This particular university identified critical gateway courses (Student Experience Courses) and tracked them apart from all other courses. The numbers below the pie chart are hyperlinks and allow the administrator to drill down to find out which students are the least active in courses, which then provides opportunity for faculty or student advisors to follow-up with those determined to be at-risk.

Figure 1: Student Activity Dashboard

Figure 2 is a comparison of student activity in two sections of the same course for a particular term, with the most intense blue indicating the least amount of interactivity and red showing the most intense student engagement. This is helpful both for individual faculty to see where they can work to more deeply engage students, as well as for curriculum developers to look for content modifications that will help foster more student interaction.

Figure 2: Student Activity ‘Heat Chart’

Returning to the data mining metaphor between retail and its potential application in education, we can now imagine how a technology-enhanced content delivery system could use data to build adaptive learning paths by student. The system could be trained to look for patterns in how students respond to intervention strategies and to then prescribe content based on aggregated results from students with similar learning profiles, who achieved significantly improved results on subsequent outcome based assessments in previous instances of a course.

Because the assessment and accountability movement is now thirty years old, most institutions are collecting data but many still struggle to effectively analyze and act on what has been gathered. Figure 2 above illustrates the application of visualization techniques to raw data in a way that provides more actionable insights to stakeholders.

Data visualization is becoming a more critical piece of the continuous improvement process on campus. For example, Figure 3 is a raw data extract showing outcome performance by students over time.

Figure 3

If a program’s goal is to have at least 70% of students achieve mastery by outcome, it would be more effective to present the data as illustrated in Figure 4.

Figure 4

When looking specifically at data’s impact on improving the student learning experience, educators essentially have two options. We can either drill down to look at individual student performance by outcome to find opportunities for improvement in formative assessments that support students with remediation options before they complete the current module or course. Alternatively we can look for more global curriculum improvement needs by evaluating achievement of program student learning outcomes across time. This second approach means it’s usually too late for current students, because the data is being evaluated after the course has ended, but the diagnosed changes can improve the curriculum for future students.

U.S. higher education has been under considerable pressure to improve accountability for student learning. With the transition to digital content distribution, the next few years are going to increasingly offer educators the opportunity to make data informed decisions that positively impact teaching and learning.

Brian Epp, M.Ed. | Assessment & Analytics Group, Academic Training & Consulting| Pearson eCollege