Tweaking the Meritocracy
Supposing you had a meritocratic state, and you wanted to adjust the academic inclinations of the populace. The best way would be to adjust the examination system so that study in specific areas would be rewarded by perceived economic or social advantage, normally conveyed by that well-established proxy, aggregate test scores.
So let's look at Atlantis. This is a state in which the young, just before they enter their teenage years, are tested to the brink of destruction (but mostly, not beyond) by a test called the Phased Stress Loading Exercise. In this test, the candidates are subjected to four kinds of load.
The first kind is linguistic; it is a double-weighted test (at least at the raw score level) designed to evaluate a candidate's competence in the language that he or she will have no choice but to use in almost every public area of life (and in 60% of the cases, every private area of life too) for the forseeable future.
The second is pseudocultural; it is also double-weighted at the raw score level, but it evaluates a candidate's competence in a language that is sometimes touted as a future trade opportunity, and which is otherwise used by less than 40% of the population. The double-weighting here is a political tool which was used to eliminate that language's dialect rivals decades ago, thus severing links between the modern population and its ancestors.
The third kind is mathematical; this is supposedly easy to test, and so it is. However, the test itself has been made to discriminate better simply because it doesn't have double-weighting at the raw level and yet is probably more important than test two. This of course has inspired furious rants from those who feel discriminated against — which is what the test is designed to do.
The last kind is scientific; this is arbitrary, as most tests of scientific ability are at that age. What it does test is scientific knowledge at a simplified level, coupled with capacity for absorbing advanced concepts at an abstracted level. This is somehow supposed to be correlated strongly to future performance in science. However, this is not true except in the general sense that students who are good at memorizing things at the age of 12 are often good at doing that at age 18.
Eventually, the final aggregate score is calculated. This is the sum of the t-scores for each test. The t-scores are calculated like this:
How do you tweak this?
Well, if you set a very tough paper, you will see record aggregate scores and be able to separate the best from the less good by a larger margin. However, it is probably easier to just adjust the marks awarded. After all, people will complain, "The paper was so tough that even I wouldn't be able to answer it!" Which of course says more about these people than it does about the paper.
A simpler paper is easier to grade, and since we haven't got enough competent markers in most populations, that's a bonus in time saved — less adjudication or moderation will be required. But you'll see lower aggregate scores and less-informed people will say, "Oh look, we've had a bad year!" and plot silly graphs which predict the downfall of society.
Actually, I have no idea how exactly any such tweaking would occur. But tweakability exists for any system, and I have just shown how this might manifest in the mythical Atlantean system. Think of it as an imaginary service for an imaginary public.
So let's look at Atlantis. This is a state in which the young, just before they enter their teenage years, are tested to the brink of destruction (but mostly, not beyond) by a test called the Phased Stress Loading Exercise. In this test, the candidates are subjected to four kinds of load.
The first kind is linguistic; it is a double-weighted test (at least at the raw score level) designed to evaluate a candidate's competence in the language that he or she will have no choice but to use in almost every public area of life (and in 60% of the cases, every private area of life too) for the forseeable future.
The second is pseudocultural; it is also double-weighted at the raw score level, but it evaluates a candidate's competence in a language that is sometimes touted as a future trade opportunity, and which is otherwise used by less than 40% of the population. The double-weighting here is a political tool which was used to eliminate that language's dialect rivals decades ago, thus severing links between the modern population and its ancestors.
The third kind is mathematical; this is supposedly easy to test, and so it is. However, the test itself has been made to discriminate better simply because it doesn't have double-weighting at the raw level and yet is probably more important than test two. This of course has inspired furious rants from those who feel discriminated against — which is what the test is designed to do.
The last kind is scientific; this is arbitrary, as most tests of scientific ability are at that age. What it does test is scientific knowledge at a simplified level, coupled with capacity for absorbing advanced concepts at an abstracted level. This is somehow supposed to be correlated strongly to future performance in science. However, this is not true except in the general sense that students who are good at memorizing things at the age of 12 are often good at doing that at age 18.
Eventually, the final aggregate score is calculated. This is the sum of the t-scores for each test. The t-scores are calculated like this:
- Take the candidate's raw score for any test, r, and find the difference between that and the average score of the population, m.
- Divide this difference, r-m, by the standard deviation (roughly, the spread), z.
- Multiply this number, (r-m)/z, by 10 and add 50.
- This gives the total test score for that test, t = 10[(r-m)/z]+50.
- Add up all the t-scores for the four tests.
How do you tweak this?
Well, if you set a very tough paper, you will see record aggregate scores and be able to separate the best from the less good by a larger margin. However, it is probably easier to just adjust the marks awarded. After all, people will complain, "The paper was so tough that even I wouldn't be able to answer it!" Which of course says more about these people than it does about the paper.
A simpler paper is easier to grade, and since we haven't got enough competent markers in most populations, that's a bonus in time saved — less adjudication or moderation will be required. But you'll see lower aggregate scores and less-informed people will say, "Oh look, we've had a bad year!" and plot silly graphs which predict the downfall of society.
Actually, I have no idea how exactly any such tweaking would occur. But tweakability exists for any system, and I have just shown how this might manifest in the mythical Atlantean system. Think of it as an imaginary service for an imaginary public.
Labels: Examinations, Testing, Tweakability
etched with acid at 10:22:00 am
5 comments
