Rene’s Assessment

As we’re in the middle of a review of the tools we use in support of assessment, I thought I’d share my analysis of the various tools that we come across. As today is a Sunday, we’ll start off with a simple one:

Classmarker
Classmarker is an online quizmaker that offers free quizzes (supported by advertisement) with upgrades (including removing the advertisement) for an additional fee.
Type: online service
Cost: Free with paid upgrades
Features: Multiple choice quiz, free text quiz or punctuation quiz.
Interoperability: None
System requirements: Any browser

The first thing I notice when registering, is that the UK doesn’t exist, although the 4 home nations do. A more serious point to note, as with many online services, is that all content (and so that includes all personal information, questions and results) will be the property of Classmarker.

The features of this service are extremely limited. While the Classmarker supports 3 question types, it only allows you to use one of those per test. Options such as randomisation, feedback and branding are all features you will have to pay for. There seems to be no way to import or export your questions.

The site seems to be built mainly around Google Adsense. The advertisement and a Google search box is present on every possible page, and that will include the ones your learners visit. Upgrading to get rid of the advertisements costs $24.95 (or 49.95 for a business account, whatever that means). But then your users will still ave to register with the service before being able to take the test. Allowing for unregistered learners to take a test will cost you $0.10 or more per learner. Not really value for money given the incredibly limited features that are on offer.

Conclusion: I really can’t see anything of value here. If you need something that is hosted for you, most survey services offer you more functionality. If you have your own space to host your assessments, even the simpler tools available wil offer more then Clasmarker.

Apologies to have to start of with such a negative review. I just stumbled across this tool today, and I thought I might as well write this up now. Do let me know if you have any comments, or perhaps sugestions for other tools I could review.

Tweet This Post

The National Academy for Engineering has been trying to identify the grand engineering challenges for this century. It obviously features several challenges in environmental sciences, artificial intelligence an virtual reality. I was very pleased, and slightly surprised, to also see Advance personalized learning as one of the grand challenges.

While the explanation seems to start of with a bit of a disappointing focus on learning styles, it then picks up with applications that I find much more interesting, such as tailored support to learners based on ubiquitous data collection of their progress. I am not quite sure this is an engineering challenge though. 99% of the technology that is needed to meet this challenge already exists. It is primarily our inability and sometimes unwillingness to implement this properly that makes it a challenge.

A good start could probably be made in the education of those who are going to be delivering this personalised learning. From what I recall from my various bits of formal teacher training, the emphasis was on a rather old fashioned model of learning. I was taught how to teach, but seldom did we learn how people learn.

A second area that needs challenging in my opinion, is regulations and management. In most institutes I have worked for, innovation was strangled by conservative financial management (where risk is a dirty word, and profits are always expected in advance to cover investments… a very peculiar idea). In many areas professional bodies also seem to work more to the detriment then the benefit of innovation. The message there often seems to be ‘do as we have always done, and you’ll be alright’.

Personal Learning definitely is one of our great challenge. But the challenge is not to invent it, or make it technologically possible. The challenge is ‘simply’ to implement it, and make it work.

Tweet This Post

I was watching this very interesting presentation by Martin Weller, called Bridging the gap between web 2.0 and higher education

| View

SlideShare Link

Something that caught my particular attention was Martins remarks about how technology challenges presuppositions on granularity, and what the consequences for the granularity of learning might be. I find the idea of more granular learning compelling, in particular in combination with personal learning (although Martin also rightfully points out that this is not just about what you want to learn, but also how you want to learn it!). On the other hand I also cannot help being concerned with what we loose from the holistic approach if we insist on atomizing everything. The whole is after all often more then the sum of its parts.

Perhaps the solution lies in a differentiation between atomized accreditations on the one side, the majority of which will probably be APEL, and separate aggregating accreditations that require you to integrate and join up what you’ve learned, and reflect on it on the appropriate level. These qualifications woudl focus more on (meta cognitive) skills and trans-disciplinary thinking.

As I’ve said before, our future is not in content, it is in guidance and accreditation!

Tweet This Post

I finally found some time to read the Cape Town OER Declaration, and a selection from the deluge of comments that have piled up in my RSS-reader the past weeks. Given the critical tone of most of these, I was expecting something very fundamentally flawed.

The declaration is an initiative of the Shuttleworth Foundation (yes, that’s the same Shuttleworth as the one in Ubuntu). The purpose of the declaration is to accelerate the international effort to promote open resources, technology and teaching practices in education. Unfortunately many advocates of open learning have not really welcomed the declaration with open arms.

A noteworthy example of this can be found in the blog Half an Hour: Criticizing the Cape Town Declaration by Stephen Downes. While I normally find Stephens post very eloquent, I cannot support many of the arguments he makes. It leaves me with the impression that his main point (and that of many others) is that they are a bit miffed of they weren’t consulted. To me the whole ‘let’s decide everything in a big all encompassing committee’ culture is exactly the reason that hardly anything ever gets done, or done properly in education. Open source communities understand that democracies don’t work. A benevolent dictator, or a meritocracy (or both) is what you need. I’m sure Mark Shuttleworth understood exactly that when he limited participation in drafting this initial declaration.

I for one support the initiative. I’m going to sign up for it now, and I would invite you to consider the same.

… Which reminds me, I still need to formally license the stuff on here with a creative commons license…

Tweet This Post

I’ve been looking into criteria for assessment technologies a lot lately. One reason is that we are looking into migrating our current system to a new platform (as the old one, Authorware, is no longer supported). The other reason is that I have been invited by the Joint Research Centre to take part in a workshop on quality criteria for computer based assessments. I will be posting on the outcomes of that workshop next week. For now though, here are some of my thoughts on the topic.

Flexibility
The main strength of our current system is flexibility. This has several aspects, that are all important in their own right:

• Flexibility in design: The layout of the question can be modified as desired, using media and such to create an authentic and relevant presentation
• Flexible interactions: There is no point in systems that have parameterised 5 question types for you, and all you can do is define a title, question text, alternatives and select the right answer. Interactions testing and supporting higher order skills are, or should be, more complex then that.
• Detailed and partial scoring: A discriminating question does not just tell you whether you were completely right, or completely wrong. It can tell you the degree to which you were right, and what elements of your answer had any value. It might also penalise you for serious and fundamental mistakes.
• Detailed feedback: A lot of mistakes learners make are predictable. If we allow assessment systems to capture these mistakes and give targeted feedback, learners can practice their skills while lecturers can focus there time on more in depth problems that require their personal engagement.
• Extensive question generation and randomisation options: For the re-usability of assessments, generating questions using rules and algorithms given a single question almost infinite re usability. On the assessment level, the same is true for assessment generation based on large banks with questions tagged with subject matter and difficulty.

So far, no real news for TRIADS users (although no proprietary system I know of really supports this well).

Questions without assessments
As Dylan Wiliam so eloquently worded at the ALT-C conference (you can find his podcast on the matter on http://www.dylanwiliam.net/), the main value in learning technology lies in “to allow teachers to make real-time instructional decisions, thus increasing student engagement in learning, and the responsiveness of instruction to student needs.” I could not agree more. However, this means that questions should not just exist within the assessment, but instead be embedded within the materials and activities. Questions become widgets that can of course still function within an assessment, but also work on their own without loosing the ability to record and respond to interaction. This, as far as I’m aware, is unchartered territory for assessment systems. Territory that we hope to explore in the next iteration of our assessment engine.

Tweet This Post