The growing datification of higher education and rampant innovations in AI and machine learning have created a variety of responses across the sector. While these reactions range from fear around the rapid advancements in LLMs, to hope for AI reducing burdensome administrative and assessment tasks, it is worth both senior managers and educators alike adopting a more critical, reflective lens to assess the landscape, improve our data literacies, and ultimately make informed decisions. (McDermott et al., 2023)
In focusing solely on the JISC’s AI in Tertiary Education report (2021), one might conclude that recent innovations in AI are not only transforming tertiary education but more investment should be made in these areas in order to keep up with the greater role technology is playing since the COVID crisis and the increased demand for digital skills. The report notes that considerations around ethics and the ensuing “debate is outside the scope of this document”. (JISC, 2021, pp. 1). The report simply outlines what is currently possible, which could be misleading to many educators as it lacks a critical component.
One of the failings of this report, and of many conversations about data and more generally, is the lack of pedagogical and human reflection. As Knox et al. (2020) examine future visions of ‘learnification’ and ‘datification’, they focus on the psychology that underpins ‘learning sciences’:
Skinner’s ‘operant conditioing’ appears as a rather bleak view of the learner as passive tabula rasa, in comparison with what has become a common sense of active and self-knowing individuals in constructivist theory.
(Knox et al., 2020, pp. 36)
The student as passive tabula rasa is a useful metaphor to adopt in this analysis, as many of the examples covered in the report purport the benefits of AI for students at a variety of levels and contexts, but do little to examine the student experience. The report does not take into account how a student’s own experiences and knowledge influence their interactions with AI, and educational technology more broadly.
This file is licensed under the Creative Commons Attribution-Share Alike 3.0 Unported license. Attribution: Sippel2707
Using examples from my own practice, I conducted basic analytics reports on students from two of my courses on Blackboard and Canvas. The analytics generated on both platforms are generally indicative of student behaviour as both courses are content-heavy, and the Universal Design for Learning course is more so in its early weeks, but the data cannot account for working practices, and ultimately privileges frequency. While some might revisit the content pages over and over again, others might work through each block in one or two sittings, and others might download the content to work offline, rarely visiting the VLE.
Students in the above course would visit the “Topics” section most frequently for course content and activities. The figures likely only represent different working practices and not true levels of engagement with the course materials. The data would not reflect the amount of time spent reading the core materials, or reflecting on the same to prepare the associated, reflective blog posts.
The analytics here are more basic again and only indicate that the course site is most heavily visited in the early weeks of the course, which is how the course is designed in the first instance. The data does reflect the amount of time spent working through the core materials and reflecting on the same, as reflection activities are completed offline with peer groups and support. Neil Selwyn (2015) notes the “tendency of digital data to remove ‘the social’ from acts of knowing” (pp. 75), building on the limits of datification in regard to the social elements of learning.
Eynon (2013) questions: ‘What kinds of learning can a student truly keep ‘private’? Does the potentially highly public and trackable nature of learning have impacts for the learning process?’ (2013, pp. 238). This is a consideration largely ignored by the JISC report. Students involved in such pilots and case studies as outlined in the report need critical digital literacy support in order to fully understand how their participation is being tracked, and who is benefitting from it. Pangrazio (2016) notes that “Learning within a techno-social system involves technical mastery and inquiry, analysis and critique.” (pp. 169), skills not directly addressed in the JISC report despite a push for further innovation due to the demands for digital skills.
Ultimately, one must consider the human experience in relation to data and AI advancements, whether from a pedagogical, psychological, or sociological lens. Data can misrepresent the nuances reflected in the learning experience, and students and educators alike should be equipped will the skills to understand how their teaching and learning experiences can be used as data points.
References:
Enyon, R. (2013). The rise of Big Data: what does it mean for education, technology, and media research?, Learning, Media and Technology, 38:3, 237-240, DOI: 10.1080/17439884.2013.771783
JISC. (2021), AI in tertiary education: A summary of the current state of play. Available at: https://repository.jisc.ac.uk/8360/1/ai-in-tertiary-education-report.pdf. (Accessed: 29 March 2023).
Bayne, S., Knox, J., & Williamson, B. (2020). Machine behaviourism: future visions of ‘learnification’ and ‘datafication’ across humans and digital technologies, Learning, Media and Technology, 45:1, 31-45, DOI: 10.1080/17439884.2019.1623251
McDermott, J., Madden, M. and MacLaren, I. (2023) “As educators, we must step up our game in the face of ChatGPT,” The Journal, 21 March. Available at: https://www.thejournal.ie/readme/chatgpt-and-universities-6018109-Mar2023/ (Accessed: 29 March 2023).
Molloy, K. (2023). Blackboard data analytics. 25 March 2023.
Molloy, K. (2023). Canvas data analytics. 25 March 2023.
Pangrazio, L. (2016) Reconceptualising critical digital literacy, Discourse: Studies in the Cultural Politics of Education, 37:2, 163-174, DOI: 10.1080/01596306.2014.942836
Selwyn, N. (2015). Data entry: towards the critical study of digital data and education, Learning, Media and Technology, 40:1, 64-82, DOI: 10.1080/17439884.2014.921628
Sippel2707 (2005) Roman tabula or wax tablet with stylus, Wikimedia Commons. Available at: https://en.wikipedia.org/wiki/Tabula_rasa#/media/File:Wachstafel.jpg (Accessed: March 31, 2023).
Weller, M. (2019) “25 Year of Ed Tech – Images,” Edtechie.net, 12 July. Available at: http://edtechie.net/25Years/resources/images/ (Accessed: March 31, 2023).
Yes,I agree wholeheartedly to your opening statements regarding the reduction of admin and assessment – using data and AI to work with teachers. I love what Bayne (2015) says about exploring ‘…how human and non-human teachers might work together in a teaching ‘assemblage’’ such an interesting and exciting idea.
You point out that ‘The report does not take into account how a student’s own experiences and knowledge influence their interactions with AI, and educational technology more broadly.’
I find the whole issue of the student ‘experience of learning’ is very much ignored in research and ed tech solutions and it’s the aspect of digital learning I find the most interesting.
I love your examples of student analytic reports and the notion of different student approaches to time and space in learning – this is so important when considering how to design and support the student learning experience, I feel, and as you point out, data analytics can never tell the whole story. However, a teacher with that sort of critical insight about student behaviour might be able to interpret the data helpfully. Would AI be able to do this? I’m not sure and as you say ‘Neil Selwyn (2015) notes the “tendency of digital data to remove ‘the social’ from acts of knowing” (pp. 75), building on the limits of datification in regard to the social elements of learning.’
Great post, as usual, Kate and lots of interesting issues raised. The screenshots and hyperlinks were really helpful.