Showing posts with label analytics. Show all posts
Showing posts with label analytics. Show all posts

Monday, March 19, 2012

The process of open educational practice, rather than the outcomes. Stuff badges!!.

Dave Wiley posted a piece on why universities will be the biggest awarders of badges.

I can see why he thinks this is the case. Educational Institutions are the ones that have for centuries been awarders of pieces of paper  for successful working with knowledge, but not necessarily for learning outcomes!!! That's the problem with formal education, resources are provided, and interactions between people taken place, but if people actually learn anything is not easy to measure. Who is to say that all these students who were so good at the exam actually learnt anything, they might just have been very good at answering exam questions? As open education practicioners, we should think about this when discussing certification and question if this model is actually the right one for open learning.

Clearly, by awarding badges to open learning episods, for instance, you actually change them  into closed courses. We already have lots and lots of those. So, why are people suddenly so obsessed with accreditation of open courses? It seems to me that at the moment the hype generated by MITx and Stanford around their 'open' courses and the idea by Mozilla to give boy scout awards to learners are at its heart, but in my view they do not only devalue Higher Education, and produce a two-tier system, one for the people who can afford the high fees who receive the 'quality learning experience' with all bells and whistles and human interaction and support, and the other people, who can't afford these and who get an open dehumanised, machine learning experience and apparently receive a token of appreciation in the form of a badge from the institution for their effort! All this as employers might value  this second rate experience from a top-tier university more than a 'quality experience' from a not so top of the range university. The market in higher education at its best (:-( !!


Of course accreditation of prior learning is not new. Over the past twenty years there have been enough challenges in providing some sort of award for open learners' efforts. Learners might use the certification they receive for their open learning episode to show institutions that they have achieved a certain level of competency. Again, its the institutions of higher education that put value (or nonvalue) to these certificates.

How important should this be in the current technology-rich climate? And, is this really, really the direction that we would like  open educational practice to develop into? Would it not be more valuable to use our energy in thinking about the learning process, and not about the external pressures for accreditation of (possible) learning outcomes? One other model would be to developing Open Educational Resources and provide access to them, but to do this in the context of open learning environments that give learners choices and control over their own learning process and learning experiences. Environments where learners will be stimulated to interact with other human beings and be critical of the world in which they live, higher education included, and where they are tempted to be analytical. And yes, technology can be used on these environments to guide learners in this, for instance learners might be able to use analytics visualizations to see where on their learning journey they are, rather than to have an HE institution make all the decisions about the value of their learning.   There are wider benfits to open learning than a certificate by an HE institution!!!

Friday, March 4, 2011

The value of analytics in an educational and learning context

I had some time to reflect on the presentations and conversations at the 1st International Learning Analytics and Knowledge Conference in Banff. There were many thought provoking presentations that gave an inkling as to the direction people interested in education, or interested in educational technology, might take analytics in the coming years. One of the figures I found most interesting, came from Abelardo Pardo, who worked with Carlos Delgado Kloos on a 'virtual machine' that they used in their research. They asked their students to install the file on their regular computer and use it for their course work, while Abelardo and Carlos could track student activity. When they looked at the browsing behavior, only 28.51% of students accessed LMS-related pages for their learning activities, while all other pages accessed were outside this institutionally controlled environment. Now of course it is quite likely that there are contextual factors that influenced this behavior, but still, it clearly points towards a finding that students only make a limited use of the institutional LMS for their learning and that if analytics are to be meaningful, they will have to include student learning activities outside the LMS. Quite some analytics presented at the conference were related to the institutional LMS. Of course this begs the question: if students only use the LMS for such a limited amount of their learning, and data on the other learning is not collected, what will be the relevance and value of carrying out analytics on this LMS environment?



Some presentations showed that analytics might be used to enhance the effectiveness and streamlining of the processes taking place in educational institutions in four ways:
1. To support the administration
2. To adapt the learning support services to make up for deficiencies in student performance.
3. To show learners their analytics in order for them to reflect on their performance and perhaps adapt their learning and learning behavior in certain ways.
4. To adapt teaching to analytics findings about student learning and learning behavior

What do I think of these four?
1. Analytics to support the bureaucracy must always be a bad thing, as analysis of data always means inputting of data, from which follows that learners and educators will have to engage in this added burden. There is enough evidence to support that the bureaucratization of university is a negative, rather than a positive development (Foucault, Reading, Delanty).
2.I like the idea that analytics might make it possible for student support services to be better matched to student needs, but coming from a background in adult education and widening access to Higher Education, I have seen my fair share of problems with using the deficiency model to support learners. I feel more comfortable with
3. the analytics model promoted by Erik Duval who runs analytics on student activities and shows the students the results. This seems more empowering to learners as it involves a need for reflection on their learning.
4. Analytics can also be run as a research tool, so teaching staff might get a better understanding of the learner experience and the problems learners might come across in order to better match their teaching. Caroline Haythornthwaite showed us some of her visualizations of communication and group forming, which highlighted insights that analytics might provide in the ties between learners in learning settings.

If the analytics are solely run on the LMS related activity and the 28.51% figure is in any way generalizable to other institutions, of course all these analytics will only tell roughly a quarter of the story. It means that people will have to start using analytics outside the institution, on the network, as Helene Fournier and I have done here at NRC in Moncton. Of course carrying out analytics on networks is not easy as people access services in a distributed environment and the analytics would be most meaningful if these could somehow be linked, perhaps by using the same identification for all of them. It would be cool, though, and could enhance the learning experience of self-directed learners, if they would be able to quickly check if they would meet their learning goals through visualizations of their activity. As that is one thing I have learned engaging in analytics: visualisation does clarify activity pretty well.

Some other developments related to linked data are the research and design of recommender systems for learning. Currently there are problems with the testing of these as large data-sets are required to ensure reliability and consistency of results as Katrien Verbert highlighted in her talk. Some other analytics-related systems are currently under development at the Open University in the UK, such as Cohere, a discourse argumentation tool that aims for depth in discussion, and iSpot, related to BBC nature programmes, that uses a novel ranking system.

A theme running throughout the conference in several of the presentations, was the ethical dimension. analytics is about human behavior and of course there are some important ethical considerations to the collection of human data. This will be another post before long.

Learning Analytics is clearly a developing field and there is still a lot to learn for all involved! Thanks again George for bringing us all together in such a wonderful location.

Wednesday, March 2, 2011

Learning about learning analytics

I have been at the 1st International Learning Analytics conference in Banff over the past two days and it has been an eye opener to the power of analytics. Not just to enhance efficiency of educational institutions, but also to help learners in their learning and researchers and educators in getting an understanding in the dynamics in class rooms and on networks. I will write more about the most interesting points later.

For now I have included the link to the presentation at the conference by Helene Fournier and me that tells a bit more about our use of analytics in our MOOC research. If you prefer a different format, Doug Clow has been tapping away on his live-blog and given a near verbatim account of the event.You can find the link here.

Monday, September 27, 2010

The use of data to advance learning in a networked environment

The #PLENK2010 subject of the week is the eXtended web. In my blog post used in the resources of this week, I highlight a number of issues related to this: The use of Intelligent data for PLE development and networked learning, the challenges of an open online networked environment for learning, and access to technology.

In this post I would like  to delve a little deeper in the first aspect on how data can be used to 1. enhance people's searches, and 2. on how data can be used by educators, or 'knowledgeable others' to enhance people's learning.

I will start with the second point first. Ken Clark for instance carried out a study to find out if 'individual teaching staff [can by], reflecting on their courses, learn anything important from examining their courses through analytics? How can this be done effectively? What do they find?'

Of course academics and researchers have researched people's learn and teaching for quite some time, but it is only recently with the introduction of LMSs and their back-office data (eg on how often people access the course, for what activity etc.) that this type of data has been analysed. It has been analysed by administrators, to find out' learning outcomes', but to me this information is fairly limited. It becomes much more rich when you also analyse the 'discourse', the text people have written for instance in discussion boards, or on Massive Open Online courses such as PLENK, other written material and artifacts produced by students and facilitators, such as blog posts, to reach a better understanding of the ways in which people teach and learn.This analysis might help to enhance future courses and future teaching, facilitation and learning. Educators can be involved in this research themselves and directly use their findings to take action to make changes to course/curriculum/interactions etc.

The second new possibility that data collection offers, is that collected data might inform people's searches: By collecting data on people's earlier learning projects, or from people's personal profile, ranking systems and recommendations could be produced that might provide people with 'smart' information, more relevant to their needs than without.

Of course there are problems with this type of data collection as the use of 'intelligent marketing' is currently showing us. Especially privacy issues have been highlighted as being problematic as before you know it people you don't want to know particular things about you, will know them and use them in ways that are not necessarily what you intended the data to be used for in the first place. So there are also  ethical issues for research.

On the other hand, the information abundance about which I wrote in an earlier post, make it near impossible for a single human being to find and analyse all information by him/herself without some type of aggregation and filtering, or communication with others.