Since my last post on assessment, Teresa has offered a retort to my ideas here. I couldn't resist picking up on a few issues that I'm sure Teresa won't be surprised me bringing to her attention.
The first point Teresa has even admitted I'd be shaking me head at. She was spot on. I am and here's why:
A student can either do something, or they can’t – a miss is as good as a mile. The only skills or knowledge that we can assume a student has are the ones they can demonstrate.My first issue with this is that whether someone can demonstrate a skill depends totally on how well that skill or competency is defined. Driving is a fairly easy situation to define what the competency is an how to assess it; there is a whole book dedicated to defining it's rules: the highway code. When you move away to anything more grey and difficult to define, this assertion falls apart.
My original example of where this is the case diversity training for Hampshire Constabulary. To even start to assess this type of training you need to set out what the competencies are. It's very tempting to qualify this in terms of what the tutor is trying to teach: "To demonstrate to students that different cultures and backgrounds can affect the way an Officer should deal with a situation" rather than what the student should learn. The problem arises when you try and define it for the student as the outcomes can be different for different people.
What you're trying to do it change beliefs and subtle behaviours in everyday encounters. How do you assess change in beliefs? Are there a set of rules that define exactly what a person's beliefs should be in this instance? I would suggest not, and that to attempt this would be very difficult. The teaching here is more about giving people a set of tools, giving people a brief set of instructions and (for want of a better word) hope that they use them properly. Here's an analogy: say you give someone a knife and show someone how to use it. You can never be sure what someone's beliefs on the knife are. Do they respect it? Do they think it's a utility rather than a weapon? Do they appreciate it's power? Will they ever forget to use it or misuse it? More importantly here, can you ever assess any of these things? People often are unsure of what their beliefs really are until they are tested in real life. Real life is often far removed from simulation or the classroom, I can't see how you can pass or fail someone on this.
Teresa makes the point that this is about the teaching, not the learning, but only a learner could even start to assess beliefs. Teresa also says that the "assessment" could be from the learners perspective and that they can be small, atomic and unconscious. I agree this is useful, but is it assessment? The though provoking image is a good example, people could challenge their initial perceptions and perhaps change them in future, but is this really assessment and do it really qualify the standard of teaching?
"depends totally on how well that skill or competency is defined": yes it does, so we need to define them well. One of my research areas.
ReplyDeleteFor any pure STEM (Science, Technology, Engineering, Maths) subject, there is a set of rules, whether it is explicit (like the highway code) or implicit (markscheme in the examiners head). I agree that this paradigm *may* not hold for subjective subjects in the arts and some humanities, but I'm not really interested in them. If they want to be fuzzy and non-commital, then they can be. Their loss.
I argue strongly against the proposed Intended Learning Outcome for the diversity training. If the training is designed to change beliefs, then it is doomed - you can't teach that and you can't assess it. And when you look at it analytically, I don't think that causing a change in beliefs is really what you want.
The constabulary wants officers *to act* as if they have those values and beliefs. Bill and Ben are both police officers; Bill is a racist, sexist, bigoted masochist, Ben is a kind, caring, sensitive gentleman.
But, if they both treat everyone they deal with with courtesy and respect, then do you care what their internal beliefs are? So long as they *always* externally behave *as if* they appriciate and value diversity etc, it doesn't matter (academically) what their internal state is.
And it is a good thing that it doesn't matter, because you can't open someone's head and have a look at what they know and believe. Can you tell me with 100% certainty that the police officer you respect the most isn't a Bill demonstrating the behaviours of a Ben?
Everything is on observable behaviours, and things that can be observed can be specified and assessed.
With the knife thing, what are you actually wanting to assess? That they respect the knife? You can't assess that, you can only assess whether they demonstrate behaviours that are considered indicative of respecting the knife.
ReplyDeleteAnd regarding the knife in real life: It would be flawed to say that "this assessment will determine whether the candidate will use knives appropriately in all situations"
Because, in order for that statement to be true, you would have to assess the candidate with the knife *in all situtations*. Not possible. The statement is flawed, not the idea of assessing behviours.
The A-Level Core 1 Mathematics exam doesn't check wether you can do the maths.
In reality, it determines the truth of: if given a stimulus (a question), the student can communicate the correct answer (against certain criteria)(and with correct working), given certain tools, in a certain context, with certain constraints.
The assessment doesn't fundimentally assess anything else. We interpret the results of the assessment and assume that a passing candidate can reproduce those results in other contexts - i.e. in the workplace. But the student could have guessed each answer, faked the working-out and passed the exam with no understanding of the mathematics whatsoever. We still give them the qualification because they demonstrated the behaviour - we don't know what is going on in their head.