Rene’s Assessment

I attended a guest lecture by Jesse Martin on how Psychology can contribute to the scholarship of teaching. It was an interesting talk, but the thing that really caught my attention concerned feedback.

Jesse was illustrating how we often fail to apply well understood psychological principles and research to our practice by looking at the current hype around feedback. According to existing cognitive research, extensive feedback actually breeds compliance, not learning. While targeted feedback at key points in someones development is important, continuous detailed feedback actually inhibits independent critical thought.While a bit of a shock at first, I must admit that this does make sense on an intuitive level. So then what do we do?

Well, there are some usefull suggestions from existing psychological research here as well. One observed phenomenon that we might exploit is the testing effect. This effect describes how neural connections are strengthened by retrieval. practice, such as through formative assessment, can therefor enhance the retention of what has been learned. it has also been shown to improve the retention over time.

So formative assessment is still seen as crucial. But it might be that the activity is much more important then the feedback resulting from it.

Tweet This Post

No of course not, would be my first response. However researchers from Bristol seem to disagree, as can be read in an article on the BBC website “Left-handers‘ lower test scores“. In the article researchers seem to conclude that the lower scores obtained by left-handers and mixed-handers mean they are more prone to cognitive developmental problems. They even advise that a test of ‘handedness’ is administered to guide early intervention strategies.

Now I haven’t had a chance to examine this research, but on the face of it this seems a bit odd. As someone with a background in computer based assessment, I am very acutely aware of validity issues. When computers are used to assess, the question ‘is this medium disadvantaging students’ is asked very regularly (perhaps even somewhat too often). It strikes me that with our pen and paper based assessments, this question is not asked often enough.

Might it be that our traditional assessment system, that has a very high emphasis on writing skills, is disadvantaging students who are not naturally equip ed to deal well with our particular written tradition?

But even if my doubts are unfounded, is pre-emptive testing really the answer to this issue? Are we going to translate this statistical trend into something that is going to stigmatise individuals without them necessarily having any related difficulties? I think that is really taking things a bit too far.

Tweet This Post

About 2 weeks ago a very interesting discussion on open accreditation started, I think on D’Arcy Norman’s blog. Some of the responses, such as for instance David Wiley’s response, are very edupunk. Do we even need degrees? I’m not sure that’s a viable position to be honest. I think George Siemens hit the nail on the head when he said that “providing a statement of competence is only value when the provider of the statement is also trusted”. Traditionally it has been institutions like our Universities that have instilled that trust. It was against this background that I have argued that accreditation is a key part of the value proposition for HE. But to be honest, I’m not so sure about that anymore.

In a draft of a call for action I read recently, Microsoft, Cisco and Intel are calling for serious reforms to our assessment system, as they feel it no longer assesses the skills that they value (creativity, collaboration and communication to name a few). That is a very serious indictment, but I think not an unjust one. many of these skills are, or should be, implicitly part of what we think of as “a degree”. But if they are not assessed, how do we ensure they are taught, and more importantly, learned? This becomes even more important when we are increasingly atomising the curriculum. If we want to let students pick and mix, we should at least be able to ensure that the sum of their choices still adds up to what we consider to be the whole of their degree.

I think a transparent and reliable way to assess these 21st century skills would go a long way to solving some of our problems in lifelong learning. It would make the accreditation of prior learning easier, as in my opinion it is this ‘hidden curriculum’ that often concerns people when considering accrediting prior learning. And with prior learning, instantly we have a vehicle to enable a flexible curriculum that spans multiple universities, or the incorporation of non-institutional learning into a qualification. But more crucially, if we can measure these things transparently perhaps trust becomes less important. If degrees no longer are black boxes with a reputation, but an open book that we can all evaluate ourselves… Portfolio anyone… ?

Tweet This Post

I have previously expressed some ideas about the value of higher education and how, at least for the less research intensive institutes, it is moving away from content and knowledge, and towards guidance and accreditation. However a few separate experiences this week have lead me to start thinking slightly differently about the future and value of higher education.

It all started with my enrolment on the connectivism course that is being prepared by Stephen Downes and George Siemens. I think it was Stephen who made the case for assessment to be individual. After all learners come to a course or activity with individual goals and ambitions, and so it doesn’t really make sense that they would be assessed in the same way. While this doesn’t invalidate the importance of assessment and accreditation, it does question the validity of having predefined outcomes and criteria for these perhaps.

over coffee this morning I had a discussion with a colleague, who was explaining to me the importance of the community of practice, and how we needed to find a way to make learners part of a community of practice before and after their actual enrollment on a module or course. He made a very strong case for what should be a major benefit of doing a course with the University: Joining a community of peers and experts. Very consistent with Stephen’s ideas I thought.

Then this afternoon, while I was wrestling the backlog in my GReader, I stumbled on a piece on the value of social networks by Engeström (via Grainne’s blog) which again confirmed this notion. Basically Engeström explains that a relation, and thus a network, only has value as a result of the object that this relation is built on. In Flickr these are pictures, in Delicious they are bookmarks. Similarly in education, these could be courses or subjects, just like my colleague was proposing with the communities of practice.

And so maybe the value of HE is not primarily around accreditation. Perhaps the most important value we can offer is the organisation and support of learning networks around subjects of interest. In that case, we have a lot of work to do…

Tweet This Post

Assessment has been in the news an awful lot lately, albeit not very positively. There is of course the whole SAT’s palava, but I will resist the temptation to comment on that. My position on this is outlines in previous posts on this blog, and I can only say that it is good to see that a lot of the momentum around this seems to be finally heading in the right direction. Its a shame we often need some sort of disaster to finally be open to change. A more surprising current issue is that of the Dyslexic student’s exams battle. Which deals with a medical student’s problems with multiple choice tests, something further clarified by the BBC in a follow-up article: Why can’t people with dyslexia do multiple choice?

The comment by the student’s solicitor that “Every professional body or employer who relies for a professional qualification, or as a promotional gateway, on multiple choice questions is heading for a fall.” is of course a bit of a joke. Quite frankly I am rather appalled by what seems like a rather misguided attempt to ‘make a splash’ at the expense of something as crucial as our exams system. While there are many gripes that you could reasonably hold against multiple choice question, I don’t think the link to dyslexia is really that valid. Considerations around presentation, or even using screen readers, can reasonably address most potential issues that might result from a disability. in addition, I think we should not shy away from critical reflection on the degree of special provisions that we put in place to accommodate students, as these provisions could significantly alter the nature of an assessment and then compromise the validity and equitability of the award. There will always be differences between learners in how well they perform in various types of assessment. This is one of the reasons to make sure there is a variety of assessment methods being used.

The more interesting question though, is around authenticity. The student in question is quoted in the article, saying that “In normal day life, you don’t get given multiple choice questions to sit. Your patients aren’t going to ask you ‘here’s an option and four answers. Which one is right?”. And to an extend I think she has a point there. While there will always be situations in which we will have to rely on ‘proxy’s’ to infer attainment, I do agree that currently we rely way too much on proxy’s that are sometimes quite remote from the competencies that we try to measure. In this sense education system is stuck in it’s traditions, in stead of applying the objective and critical reflection that we say we value so much in higher education.

A similar point, and some suggestions for moving forward, are made in the blog post 21st Century Assessment, where this ‘formula’ is proposed for a modern fit-for-purpose assessment system. Especially the elements of collaboration and peer assessment are extremely important and very much underutilised in our current practice. Partly I suspect that this links in with how uncomfortable we still are with the loss of our position as the holder and tranferrer of all knowledge. This role warranted a 1 to many broadcast model of education. Education today however is moving much more towards a many to many model, whereby the role of the teacher is much more one of guidance, coaching and accreditation of a learning process that involves peers, external resources and actors and experiences from previous professional roles. I’m not quite sure we are really ready to fulfil that role yet though.

Tweet This Post