Rene’s Assessment

I must first of all thank Stephen Downes for his truly excellent post on the  Monkeysphere Ideology. It is a profound and insightful analysis of some unhealthy fundamentals in society, of which this banking crisis is perhaps just a single symptom. This post is about how the education sector perhaps is suffering the same symptoms, and so might actually be headed for a similar meltdown.

Fail

It is argued that the financial crisis is one of destructive incentives, and hyperinflation of quality ratings, that lead to a massive over-investment in what turned out to be worthless assets. A truly horrific oversimplification, but it will do for the purposes of this post. Higher Education also rates it’s products, and ironically it is a credit rating system. Attainment in the UK is measured in academic credit on levels 4 (first year undergraduate) to 8 (doctorate). Credit on these levels is highly valued, and often seen as key to successful participation in the modern information society. As a result there is a dramatic effort to widen participation in an attempt to get up to 40% of the population equiped with a level 4 qualification or higher. And in a similar way to how mortgages were handed out without ensuring clients had sufficient capital and earnings for repayment, we are now handing out academic credit without ensuring that when we hand it out, it represents adequate intellectual capital.

There is no need for this, as my case certainly doesn’t revolve around some elitist notion that the majority of the population is fundamentally incapable of learning and functioning on this level. The problem is that we are being given the wrong incentives. We are incentivized to pass, preferably with a first. Our funding, and our ranking in league-tables depends on it. And slowly but surely, this erosion is also eroding the value of Higher Education. And unfortunately, it erodes the value of all degrees, just as the credit crisis is devaluing all our houses, regardless of our credit-worthiness. And with the value of qualifications diminishing, and their cost increasing, inevitably the time will come that this eroded qualification is no longer worth the inflated tuition fee that is being asked for it.

The key for recovery to me is failure. We should be allowed to fail, learn from that failure, and let a phoenix arise from it’s ashes. We seem to have missed the point that success is a value that is relative, relative to failure. We forget that the greatest of successes have often come from strings of failures. Who knows what great innovative company might arise from the ashes of a bankrupt General Motors, but we will probably never know. Just like we will never know how great some of our students might have been after overcoming criticism, setbacks and failure, because in stead we have sent them away with only a marginal and ever decreasing  successes. Failure is the foundation of greatness, and we need to learn to embrace it.

Tweet This Post

I watched a talk from Barry Schwarts yesterday. The talk (see below) was dressed up in a lot of obama and economic downturn, but deep down there were some very valuable truths about education.

I completely agreed with the notion that we are killing excellence, compassion and creativity by insisting on conformity and procedure, “health and safety” being the favorite example on every body’s list I would imagine. The arguments in favor of rules and incentives are often hard to counter, but I think Barry did this exceptionally well.

Rules might prevent mishap, but they guarantee mediocrity

Incentives ensure that any decision is made with only self interest in mind.

I think there is a lot to be learned from those 2 statements, in particular when it comes to our assessment system. A lot of people might think Denis Rancourt is taking things a bit too far by giving all his students an A+ at the start of their module, but I do think there is something to be said for taking away the abundance of rules (the assessment criteria and marking scheme) and the incentives (the grades) to then unleash true unburdened curiosity and learning. Would we rather our students learn a little that we feel we can quantify and evidence, or a lot that we have less control over? Is it the degree that has value, or what has been learned?

Tweet This Post

This weekend I received the outcome of my application for certified membership of the Association of Learning Technology. The scheme, CMALT, was set up with the intention of helping to create a professional identity and community for learning technologists. While I am pleased to say that the outcome of my application was positive, I do still wonder about the most appropriate way to implement professional development and accomplishment, in particular in education.

Certification can signify accomplishment and proficiency, but it also forces a level of compliance with the profession’s set of systematic knowledge and practices,  sometimes  at the expense of innovation and creativity. The recognition of accomplishment and proficiency is desirable by professionals, while the standardisation and enforcement of established practice is valuable for clients. But in education, the equation is slightly different.

Educators tend to have fairly little drive to gain additional qualifications and certifications. Perhaps this is because they are already highly qualified professionals. Perhaps it is because they realise better then most what the limitations are of assessment and accreditation. More importantly though, educational practice also has very little in terms of established codified practice. Teaching is more a matter of personal preference and style then it is the result of any commonly agreed body of work, whether scientific or experiential , and so you could wonder if it actually qualifies as a profession.
I’m not sure yet how to qualify this problem. On the one hand I am very wary of the sometimes incestuous culture of conformity that professionalisation can result in. On the other hand I am also convinced that education would benefit from a greater transparency and accountability of adopted practices. Where the balance lies, I do not yet know.

Tweet This Post

Scientific, by Brittany G

I’m currently reading The Black Swan: The Impact of the Highly Improbable by Nassim Nicholas Taleb. It’s a refreshing and humbling account of our inability to understand randomness. Two of the arguments in the book stuck me as being very relevant to assessment.

The first of these is an urgent reminder of what Karl Popper taught us about the scientific method: A million confirmations of a thesis will not yield us much information, as we might have just missed the million-and-first circumstance in which the thesis does not hold. However a single falsification of the thesis will establish that is is untrue.  And so we can only really learn through falsification, not justification.

A summative assessment is really a test of the hypothesis that our learners are  competent. With this in mind, good scholarly practice dictates that we test this hypothesis by finding possible weaknesses in it, and attempting to expose them. So maybe we should not be looking for learners to demonstrate their competence, but we should be attempting to let them reveal any incompetence they might have.

With this in mind, there is perhaps reason to question certain assessment practices such as negotiated assessments, essays and portfolios. These all allow the learner to demonstrate competence largely on their own terms, without allowing us to test for incompetence on ours. It is the learner who takes the lead here in what to put on display for our prying eyes, which also provides the opportunity to perhaps not display the more questionable areas of their attainment. While from a motivational standpoint this has many benefits, it does seem to contradict the basic principles of the scientific method.

There are other approaches that seem better aligned with this principle. Exams are an obvious choice. But I also think a well designed problem based assessment can work in accordance with scientific principles, as long as the assignment is designed with the right pitfalls and challenges that could trigger falsification. A viva voce also provides an excellent opportunity for falsification, provided that the examiners question the candidate appropriately.

Is this something that should inform our practice? Is an assessment regime without a significant component focussing on falsification actually valid, or are we subjecting ourself to a fallacy?

The second topic links to how our behaviour is very domain specific, and what this might imply for assessment practice. I will save that for another post.

Tweet This Post

About 2 weeks ago a very interesting discussion on open accreditation started, I think on D’Arcy Norman’s blog. Some of the responses, such as for instance David Wiley’s response, are very edupunk. Do we even need degrees? I’m not sure that’s a viable position to be honest. I think George Siemens hit the nail on the head when he said that “providing a statement of competence is only value when the provider of the statement is also trusted”. Traditionally it has been institutions like our Universities that have instilled that trust. It was against this background that I have argued that accreditation is a key part of the value proposition for HE. But to be honest, I’m not so sure about that anymore.

In a draft of a call for action I read recently, Microsoft, Cisco and Intel are calling for serious reforms to our assessment system, as they feel it no longer assesses the skills that they value (creativity, collaboration and communication to name a few). That is a very serious indictment, but I think not an unjust one. many of these skills are, or should be, implicitly part of what we think of as “a degree”. But if they are not assessed, how do we ensure they are taught, and more importantly, learned? This becomes even more important when we are increasingly atomising the curriculum. If we want to let students pick and mix, we should at least be able to ensure that the sum of their choices still adds up to what we consider to be the whole of their degree.

I think a transparent and reliable way to assess these 21st century skills would go a long way to solving some of our problems in lifelong learning. It would make the accreditation of prior learning easier, as in my opinion it is this ‘hidden curriculum’ that often concerns people when considering accrediting prior learning. And with prior learning, instantly we have a vehicle to enable a flexible curriculum that spans multiple universities, or the incorporation of non-institutional learning into a qualification. But more crucially, if we can measure these things transparently perhaps trust becomes less important. If degrees no longer are black boxes with a reputation, but an open book that we can all evaluate ourselves… Portfolio anyone… ?

Tweet This Post