Rene’s Assessment

Certainty based marking (sometimes erroneously refered to as competence based marking) is an advanced scoring strategy that requires learners to classify how certain they are of their response when submitting it. A higher certainty carries a possible higher reward, but also a much higher penalty when the response is incorrect. As such certainty based marking can mitigate guessing on constrained response items in an assessment, but it is also ver useful as a stimulans for reflection. More information can be found in articles like “Certainty-Based Marking (CBM) for Reflective Learning and Proper Knowledge Assessment“.

There are other interesting options to explore however, and I was reminded of one when I read
Conscious Competence – a reflection on professional learning, which talks about the conscious competence model. In my opinion, these two fit together very nicely, as depicted in the diagram below. Candidates providing the wrong answer, but indicating a high degree of certainty about their answer can be considered as ‘unknown incompetent’, as they seem unaware of their misconceptions. Candidates providing the wrong answer with a very low degree of certainty have progresssed to ‘known incompetence’, as they have at least correctly identified their lack of understanding. When providing the correct answer with a low degree of certainty, learners can be assigned to the unknown competence stage untill finally they progress to known competence if they provide a high certainty correct response.

Although I am still looking for an opportunity to actually try this in practice, I think it has a lot of potential in spporting an integrated formative and summative assessment strategy.

Tweet This Post

I just returned from our annual visit to Lougborough for the CAA conference. It was good to catch up again with colleagues from the UK and abroad. It’s always interesting to see themes emerging from amongst the vast number of interesting papers and sessions. It however also worrying how some things seem to change very little over time. I think it was Denise Whitelock from the OU who rightfully pleaded for the education sector to grab control of e-assessment. Because there still aren’t any good assessment systems. The commercial systems in general are pedagogically poor, and the HE sector seems to have a very difficult time producing anything beyond prototypes, or very discipline specific an narrow innovations. Nevertheless, there are lots of interesting things happening.

CAA and language

There has been a lot of activity in the use of advanced technology to support languages. We have briefly looked at the use of speech recognition and text to speech technology to support ESOL this year, and so I was very happy to see similar interests and developments in various other places. Xin Yu and John Lowe from the University of Bath are investigation the use of recorded audio and video in the assessment of spoken English in several Universities in China, where apparently a basic mastery of English is a mandatory part of the curriculum for all degree programs. Cambridge Examinations presented their new online assessment environment, but I must say that pedagogically I found little of interest there. The solutions main focus is to cope with the almost industrial scale on which they assess students. What did sound interesting was the research done by the University of East Anglia, the SQA and the RNID to use avatar signing in assessment. The avatars used are quite advanced, in order to accommodate for the rich set of expressions they need to properly convey sign language. I would be really interesting to see if we can use these avatars in our own developments in the ESOL domain.

Facilitation of marking and reflection

Another interesting development is the increased use of technology in support of reflective processes, such as evaluation and peer review. I think e-assessment has suffered greatly by the association with MCQ quizzes, and so it is good to see that there are a lot of people waking up to the realisation that the value of technology is not (necessarily) automation. In fact I would say that these types of facilitation are usually more innovative and transformative then the common automation of existing practice. This was also confirmed by Bobby Elliot from the SQA, who referred to this practice as assessment 1.5 (as opposed to assessment 2.0 .. and we all want everything to be 2.0 these days of course).

Tweet This Post