Abelardo Pardo, co-coordinator of the learning analytics SIG, has just published a new paper in The Journal for Assessment and Evaluation in Higher Education. This paper explores feedback in education and how this can play out and be extended in data rich learning environments.
(Link to paper)
Courtney: In this paper you explore ways to improve feedback in education. What is feedback? And what makes good feedback?
Abelardo: There is quite a lot of literature about the definition of feedback. For the purpose of the paper, and based on previous definitions, I propose to define it as "a process to positively influence how students engage with their work in a learning experience so that they can improve its overall quality with respect to an appropriate reference and increase their self-evaluative capacity."
Good feedback is a very elusive concept, but it could be described when this process is effective, that is, it influences students in their work in a positive way. In my experience feedback is most effective when it is a conversation that promotes students thinking about how they approached a task, what can be improved, and how it can contribute to better evaluation capacity in the future.
In my experience feedback is most effective when it is a conversation that promotes students thinking about how they approached a task.
Courtney: Teachers normally give feedback based on personal observations of their learners. How does learning analytics interface with such feedback?
Abelardo: Feedback is a process that needs the participation of students. Teachers may provide information, but if students are not influenced by it, it is not feedback. The data extracted from technology mediation can provide very good support for the overall process. We tend to think of feedback as something teachers do once an assessment is submitted. I propose to think about it in terms of a process that can benefit from the insight derived from data collected while students participate in a learning experience.
Courtney: What is a data-rich learning environment?
Abelardo: Whenever we have technology mediation, it is very likely we leave traces of our engagement. For example, if we write a document in a platform that allows collaboration and comments, it is possible to capture additional information about how a document was created. This data may provide very valuable insight about the process instead of the final product. Assessment typically focuses on the final product. Making the process partially visible, may have a significant impact on the provision of feedback.
We tend to think of feedback as something teachers do once an assessment is submitted. I propose to think about it in terms of a process that can benefit from the insight derived from data collected while students participate in a learning experience.
Courtney: Will our learning experiences become increasingly data-rich in the future? Is this a good thing?
Abelardo: I certainly anticipate it. The challenge though is how to blend the presence of data with a learning design that is being enacted, and identify the best way to use data to improve aspects such as the feedback process. The reason why feedback is an interesting aspect is because it helps to focus the use of data on something that is directly related to the pedagogical strategy. In other words, it guides the use of data towards something that has a positive impact in the learning experience. It is important to have a robust focal point in the learning experience to then guide the use of data (and not the other way around).
Courtney: Where can people go to find out more about learning analytics?
Abelardo: There is an increasing number of publications appearing in journals. The Journal of Learning Analytics would be a good starting point, but there are contributions in many others. I recommend reading publications that combine the use of data with a solid pedagogical underpinning. Additionally, the upcoming 8th edition of the International Conference in Learning Analytics and Knowledge will take place for the first time in Australia on 5-9 March in Sydney. Call for papers closes 2 October.
As for using these techniques in your own teaching, I would recommend to start thinking about the data that is already available to you and how to connect it with a concrete aspect of the learning experience. Then, you may widen the scope to more sophisticated data sources but always connected to concrete learning aspects.