Written by Brian Epp, M.Ed., Assessment and Analytics Group Supervisor, Academic Training and Consulting, Pearson
Learning analytics (LA) is one of the most discussed trends in education technology today. EDUCAUSE’s Next Generation Learning Initiative Challenges defines it as “the use of data and models to predict student progress and performance, and the ability to act on that information” (Brown, 2011). At its heart, “analytics marries large data sets, statistical techniques, and predictive modeling” (Campbell, DuBlois, & Oblinger, 2007). Common to all definitions in the literature is the idea that the amount of digitized information available is vast and comes from a wide variety of systems.

While integrated, workable solutions are still in their nascent stages, the foundation for learning analytics began with the birth of online learning and the advent of the Learning Management System (LMS) in the late 90s. This article summarizes the opportunities and challenges for learning analytics as it applies to higher education.
The ability to act on gathered data is the central theme and opportunity of learning analytics. While also a challenge that will be addressed later, institutions have the ability to gather data from the LMS, the student information system (SIS), registrar, library, and financial aid records, social networks, admissions files, and even clicker systems for ground based classrooms (Campbell et al., 2010, Siemens, 2010, & Johnson, Smith, Willis, Levin, & Haywood, 2011). If data analysts and the appropriate subject matter experts are able to access all of this data from a single data warehouse, the correlations that can be discovered are exciting and have the ability to radically change educational effectiveness and efficiency.
At risk student monitoring and subsequent intervention strategies are the most common application for learning analytics in higher education. A recent Education Week article likened the algorithms that are being developed for student retention to credit scores used by the financial industry or to insurance premium formulas that have been developed to determine rates based on driver demographic data and driving history (Sparks, 2010). As systems become more sophisticated, content developers will be able to diagnose a student’s strengths and weaknesses, identify a student’s preferred learning style, and prescribe content, remediation, and assessment activities specific to an individual.
Technology is critical here because faculty can only do so much to tailor learning for individual students. Brown points out that LA can help point out what works and what doesn’t “at a much finer level of granularity than even before, even while a course is in progress” (2011). He goes on to say that enterprise integration gives us the ability to show students how their performance on assignments compares with their classmates by demographic category on the fly as they progress through content. There are many instances of publisher content that are used by faculty worldwide which also has the potential to “provide composite views of student learning for an entire class of institutions” (Brown, 2011). Imagine faculty being able to compare their students’ performance on standard content against other institution types (tier 1 research to community colleges), or to break it down further by profit/not-for profit status or by geographic region.
Additional LA uses in higher education include predictive models used by admissions departments to determine which students are most likely to succeed at an institution or algorithms used by development offices to determine which donors are mostly likely to contribute to fundraising requests (Campbell et al., 2007).
Despite the substantial benefits for Learning Analytics, significant challenges remain for this growing field. Data security, ownership, and privacy represent the first major hurdle to overcome (“7 Things You Should Know,” 2010). Some argue that students should have the opportunity to opt out of any system that tracks their behavior, however, the validity and reliability of conclusions that come out of academic analytics are only as strong as they are complete so having a significant percentage of students being excluded from formulas is problematic.
In addition, there are ethical concerns about the conclusions that are drawn from LA initiatives. From the “profiling” argument on one side that could be construed as labeling students before they’ve had the chance to prove otherwise, to not taking action when a formula identifies a student as at risk on the other side (“7 Things You Should Know,” 2010). These issues will sort themselves out one way or another as LA matures.
A second major challenge is finding a data warehouse that provides researchers with a single interface to query for correlations between the disparate systems that were identified earlier. Because a majority of student data lives in LMS and SIS systems, these companies are working with institutions on integrated data warehouse projects, however, it involves time and a significant commitment of both human and financial resources to complete successfully. An additional barrier to consider here is the data management politics often present on campus between the IT and Institutional Research siloes.
The third item to overcome for a successful learning analytics initiative is the issue of finding the right people to make it work and figuring out where they reside in an institution’s organizational hierarchy. Campbell et al.point out that the types of expertise required for a successful Learning Analytics project include “database administrators, institutional researchers, programmers, and domain specialists” (2010). Territorial issues can crop up with this challenge as well because pride and ego for individuals in this list of specialists can lead to isolated efforts that don’t incorporate the expertise of others.
While the challenges are real, the opportunities promise to bring some of the most compelling changes to the nature of education over the next ten years, especially given the growing accountability demands from the public for documented results. Online and blended learning environments are here to stay and have exponentially expanded the volume of data being collected about students and how they engage with content, their peers, and with faculty. As this technology matures, it will get increasingly more efficient at maximizing the diagnosis of where a learner is and prescribing tailored content for knowledge acquisition and remediation with limited faculty intervention. This will allow faculty to spend more time providing quality feedback and support to students as they submit assignments designed to prove their competency for the required course or program learning outcomes.
References:
7 Things You Should Know About Analytics. (2010). EDUCAUSE Learning Initiative. Retrieved from http://net.educause.edu/ir/library/pdf/ELI7059.pdf
Brown, M. (2011). Learning Analytics: The Coming Third Wave. EDUCAUSE Learning Initiative Brief, April 2011, 1-4. doi:ELIB1101
Campbell, J. P., DeBlois, P. B., & Oblinger, D. G. (July/August 2007). Academic Analytics: A New Tool for a New Era. EDUCAUSE Review, 42(4), 41-57. Retrieved from http://www.educause.edu/EDUCAUSE+Review/EDUCAUSEReviewMagazineVolume42/AcademicAnalyticsANewToolforaN/161749
Johnson, L., Smith, R., Willis, H., Levine, A., and Haywood, K., (2011). The 2011 Horizon Report. Austin, Texas: The New Media Consortium.
Siemens, G. (August 25, 2010). What are Learning Analytics? ELEARNSPACE. Retrieved from http://www.elearnspace.org/blog/2010/08/25/what-are-learning-analytics/
Sparks, S. D. (2010). Schools Find Uses for Predictive Data Techniques. Education Week, 30(36), Retrieved from http://www.edweek.org/ew/articles/2011/06/22/36analytics.h30.html?tkn=PYUFH5imRxQnSzgCu8%2BBlTZDvTqdfw%2FxpgF3&cmp=clp-edweek&bcsi_scan_2ABCB1C426625F76=QLgvC8wvcwAT/XcjWj3vT6kWEUcEAAAAihhqAQ==&bcsi_scan_filename=36analytics.h30.html