Come And See

For a faith that is assumed to deal mostly with intangibles, Christianity is unusually empiricist. I am struck by the frequency with which characters in the Bible are exhorted to "come and see" — that is, to make themselves physically present in order to observe a specific phenomenon.

More than that, the concept of controlled testing is not alien to the book either. In the first chapter of Daniel, the eponymous author proposes a clinical trial (well, not double-blind, but a simple one) in which he and his friends will eat a vegetarian diet for ten days and then be compared to the rest of the cohort in terms of fitness to serve in the king's court. Later, King Nebuchadnezzar very cleverly demands that his dream-interpreting sorcerers and diviners should tell him the contents of his dream first before they interpret it. Only Daniel passes this test.

It's interesting to see also Elijah's test on Mount Carmel, where he demands that his sacrifice be wet down with water three times, the water overflowing into a trench, before calling down fire from heaven. The precision of the resulting strike is now a matter of legend, and the consequences to his opponents proportionately dire.

One verse that is often quoted out of context is: "You shall not put the Lord your God to the test." That kind of testing is the test of provocation, not the test of hypothesis; it is the kind of testing we imply when we say, "Do not test my patience." The God of the Bible doesn't otherwise seem to mind having his power being tested for a specific reason.

Generally, the many tests of power in the Bible are rooted in results — a test is proposed with a result that allows falsification (i.e. proof that the assertion tested is false). It is thus no wonder that early scientists like Galileo and Newton were able to investigate the natural world without seeing conflict with their faith (although they might have cut it fine with their nominal co-religionists).

To this day, just as with science, two kinds of tests remain in everything Christian — the test of reason and the test of findings. If hermeneutic reasoning shows an assertion to be theologically unreasonable, or if the findings of an empirical test show an assertion to be false, the penalty used to be death by stoning or the sword. In this supposedly more enlightened age, we have a much, much more tolerant response to false prophets and other snake-oil salesmen of various persuasions.

Assessment and Evaluation, Knowledge and Mind

People have different philosophies of assessment and evaluation. The kind of philosophy you have depends a lot on your theory of knowledge, and perhaps your theory of mind.

Here are some crude approximations. A theory of knowledge is your concept of what it means to know, and what it means for something to be (usefully or otherwise acceptably) called 'knowledge'. [Good starting link here] A theory of mind is your concept of what mind is, how we conceptualise mental states, and how we decide what these mental states are. [Good starting link here.]

It follows that if your idea of knowledge is ontological — derived axiomatically through reason but without 'real' evidence — then you would test for abstract knowledge. It is true, so all you have to assess is whether the candidate knows it, and perhaps whether the candidate knows why it is logically necessary to believe in it (e.g. by mathematical or theological proof from basic principles or axioms).

It also follows if your idea of knowledge is empirical — derived from observation and collation of 'real-world' data — then you would test for concrete knowledge. It is true only if it has reliable existence which can be detected by an observer. So you would need to assess whether the candidate can produce what is required (e.g. by making a pot, by swimming 50m, by making up a 1.0M solution of sulphuric acid).

Very often, we have conflated our idea of knowledge so much that we have no idea what to test, and the act of a candidate writing down an ontological proof is taken as empirical proof of reality. We thus evaluate the candidate as intelligent despite knowing full well that we can't tell. Which brings us to the theory of mind.

If you believe that mental states are not only apprehensible but comprehensible, it is quite likely you believe so by congruence. In other words, you believe that if a person does X, that person would be thinking much as you would be if you were doing X. Some people get this right more often than others, and they are said to be high-empathy or to have high EQ. Some people get this wrong more often than others, and we think of them as psychotic or low EQ (or otherwise deficient).

This is why when students do things that our tests don't rate highly, we think of them as not having appropriate intelligence or preparation or skill (or whatever). That's if we're being cautious and/or charitable. Many of us would just say, "This candidate is low-performing." (Or 'dumb', or 'stupid', except that nowadays we only think it but try not to say it.) Our concepts of high performance depend on how difficult it would be for us, the educators, to carry out. What's easy for us should be easy for those we have taught, right?

Wrong, of course. In my ramblings on this blog, I've pointed out the circularity of our ideas about intelligence and its vulnerability to subjective evaluation even when considered by experienced educators and scientists.

In effect, we need to stop thinking of intelligence as a scored function — high-scoring, low-scoring or whatever-scoring. We need to think of it as a) what jobs need to be done, b) what are the ways they can be done, c) does this person figure out such ways quickly? We can omit b) if we just want to see whether a person can get the job done, regardless of whether we are able to figure out how it might be done. We need also to think of it as d) can our candidate figure out what jobs need to be done?

You see, we have no ability to actually get into someone's head, short of telepathy. We cannot tell how cognitively brilliant someone is except by whether that person gets the job done fast. We can plausibly rate candidates on how convincing their products are to us, but that would be subjective — the more their thinking corresponds with what we think is good, the more points they get. All these are old criticisms.

Moving forward, I suspect what we should do is empower candidates by teaching them every tool we know, how to evaluate the usefulness of tools, and how to create their own tools. A lot of that teaching will comprise meaningless exercises. We must be honest and tell them that we don't know how valuable the education we're giving them is. When we are long gone, they will have to prove those tools in the crucible of reality, and our hope should be that they will survive it.

All Is Vanity

Vanity is a curious kind of failing. It strikes in unexpected places, at unexpected times. Very few are immune to it, no matter how wise. There is always the little voice in the head that says, "There but for the grace of God go I," and immediately, we stumble. Today I was reading from one of the books of the wise, and mourning the fact that so many of us are like this, so many things are like that, and everything in the end is indeed vanity.

Sometimes, we read things into events and into writings that are just not there at all. Recently, some controversy arose about whether President Bush was pointing a subtle finger at Senator Barack Obama when he spoke about appeasement before Israel's Knesset. To some, it was obviously so and a terrible thing to have done; to others, it was not so obvious, especially since the obvious reference was made about a Republican Senator and not a Democrat. All of that was quickly put into a more human perspective (although it may very well come back on the global front) by the news of Senator Ted Kennedy's malignant brain tumour. I observed all this through the daily updates of the New York Times.

The whole course of events reminded me about the much older controversy surrounding Carly Simon's (in)famous song, You're So Vain. In that song, she says the immortal lines:

You're so vain, you probably think this song is about you.
You're so vain, I'll bet you think this song is about you...
Don't you? Don't you?

The problem is of course that the answer to those lines might be, "Yes," and that answer might be true for at least one person. It is interesting though that people interviewed Ms Simon over a 35-year span to actually find out who the true subject of the song was. I think that's a valid approach; if the song's originator can't tell you, who can?

To this day, remembering Ms Simon's song and its lyrics, I try very hard not to 'read myself into things' when someone writes something that would offend me if I were indeed the target of those writings. I try to be a skeptical empiricist about it. And perhaps, that kind of thinking is indeed the lesson to be learnt in the book of the Preacher, who was King in Jerusalem.