Assessment and Evaluation, Knowledge and Mind
People have different philosophies of assessment and evaluation. The kind of philosophy you have depends a lot on your theory of knowledge, and perhaps your theory of mind.
Here are some crude approximations. A theory of knowledge is your concept of what it means to know, and what it means for something to be (usefully or otherwise acceptably) called 'knowledge'. [Good starting link here] A theory of mind is your concept of what mind is, how we conceptualise mental states, and how we decide what these mental states are. [Good starting link here.]
It follows that if your idea of knowledge is ontological — derived axiomatically through reason but without 'real' evidence — then you would test for abstract knowledge. It is true, so all you have to assess is whether the candidate knows it, and perhaps whether the candidate knows why it is logically necessary to believe in it (e.g. by mathematical or theological proof from basic principles or axioms).
It also follows if your idea of knowledge is empirical — derived from observation and collation of 'real-world' data — then you would test for concrete knowledge. It is true only if it has reliable existence which can be detected by an observer. So you would need to assess whether the candidate can produce what is required (e.g. by making a pot, by swimming 50m, by making up a 1.0M solution of sulphuric acid).
Very often, we have conflated our idea of knowledge so much that we have no idea what to test, and the act of a candidate writing down an ontological proof is taken as empirical proof of reality. We thus evaluate the candidate as intelligent despite knowing full well that we can't tell. Which brings us to the theory of mind.
If you believe that mental states are not only apprehensible but comprehensible, it is quite likely you believe so by congruence. In other words, you believe that if a person does X, that person would be thinking much as you would be if you were doing X. Some people get this right more often than others, and they are said to be high-empathy or to have high EQ. Some people get this wrong more often than others, and we think of them as psychotic or low EQ (or otherwise deficient).
This is why when students do things that our tests don't rate highly, we think of them as not having appropriate intelligence or preparation or skill (or whatever). That's if we're being cautious and/or charitable. Many of us would just say, "This candidate is low-performing." (Or 'dumb', or 'stupid', except that nowadays we only think it but try not to say it.) Our concepts of high performance depend on how difficult it would be for us, the educators, to carry out. What's easy for us should be easy for those we have taught, right?
Wrong, of course. In my ramblings on this blog, I've pointed out the circularity of our ideas about intelligence and its vulnerability to subjective evaluation even when considered by experienced educators and scientists.
In effect, we need to stop thinking of intelligence as a scored function — high-scoring, low-scoring or whatever-scoring. We need to think of it as a) what jobs need to be done, b) what are the ways they can be done, c) does this person figure out such ways quickly? We can omit b) if we just want to see whether a person can get the job done, regardless of whether we are able to figure out how it might be done. We need also to think of it as d) can our candidate figure out what jobs need to be done?
You see, we have no ability to actually get into someone's head, short of telepathy. We cannot tell how cognitively brilliant someone is except by whether that person gets the job done fast. We can plausibly rate candidates on how convincing their products are to us, but that would be subjective — the more their thinking corresponds with what we think is good, the more points they get. All these are old criticisms.
Moving forward, I suspect what we should do is empower candidates by teaching them every tool we know, how to evaluate the usefulness of tools, and how to create their own tools. A lot of that teaching will comprise meaningless exercises. We must be honest and tell them that we don't know how valuable the education we're giving them is. When we are long gone, they will have to prove those tools in the crucible of reality, and our hope should be that they will survive it.
Here are some crude approximations. A theory of knowledge is your concept of what it means to know, and what it means for something to be (usefully or otherwise acceptably) called 'knowledge'. [Good starting link here] A theory of mind is your concept of what mind is, how we conceptualise mental states, and how we decide what these mental states are. [Good starting link here.]
It follows that if your idea of knowledge is ontological — derived axiomatically through reason but without 'real' evidence — then you would test for abstract knowledge. It is true, so all you have to assess is whether the candidate knows it, and perhaps whether the candidate knows why it is logically necessary to believe in it (e.g. by mathematical or theological proof from basic principles or axioms).
It also follows if your idea of knowledge is empirical — derived from observation and collation of 'real-world' data — then you would test for concrete knowledge. It is true only if it has reliable existence which can be detected by an observer. So you would need to assess whether the candidate can produce what is required (e.g. by making a pot, by swimming 50m, by making up a 1.0M solution of sulphuric acid).
Very often, we have conflated our idea of knowledge so much that we have no idea what to test, and the act of a candidate writing down an ontological proof is taken as empirical proof of reality. We thus evaluate the candidate as intelligent despite knowing full well that we can't tell. Which brings us to the theory of mind.
If you believe that mental states are not only apprehensible but comprehensible, it is quite likely you believe so by congruence. In other words, you believe that if a person does X, that person would be thinking much as you would be if you were doing X. Some people get this right more often than others, and they are said to be high-empathy or to have high EQ. Some people get this wrong more often than others, and we think of them as psychotic or low EQ (or otherwise deficient).
This is why when students do things that our tests don't rate highly, we think of them as not having appropriate intelligence or preparation or skill (or whatever). That's if we're being cautious and/or charitable. Many of us would just say, "This candidate is low-performing." (Or 'dumb', or 'stupid', except that nowadays we only think it but try not to say it.) Our concepts of high performance depend on how difficult it would be for us, the educators, to carry out. What's easy for us should be easy for those we have taught, right?
Wrong, of course. In my ramblings on this blog, I've pointed out the circularity of our ideas about intelligence and its vulnerability to subjective evaluation even when considered by experienced educators and scientists.
In effect, we need to stop thinking of intelligence as a scored function — high-scoring, low-scoring or whatever-scoring. We need to think of it as a) what jobs need to be done, b) what are the ways they can be done, c) does this person figure out such ways quickly? We can omit b) if we just want to see whether a person can get the job done, regardless of whether we are able to figure out how it might be done. We need also to think of it as d) can our candidate figure out what jobs need to be done?
You see, we have no ability to actually get into someone's head, short of telepathy. We cannot tell how cognitively brilliant someone is except by whether that person gets the job done fast. We can plausibly rate candidates on how convincing their products are to us, but that would be subjective — the more their thinking corresponds with what we think is good, the more points they get. All these are old criticisms.
Moving forward, I suspect what we should do is empower candidates by teaching them every tool we know, how to evaluate the usefulness of tools, and how to create their own tools. A lot of that teaching will comprise meaningless exercises. We must be honest and tell them that we don't know how valuable the education we're giving them is. When we are long gone, they will have to prove those tools in the crucible of reality, and our hope should be that they will survive it.
Labels: Assessment, Education, Empiricism, Epistemology, Evaluation, Intelligence, Odd Thoughts, Ontology
etched with acid at 12:33:00 pm
1 Comments:
Thank you sir! (:
Post a Comment
<< Home