Gaming the System: A Brief Introduction
The rise of Atlantis has always been predicated on quantitative sorcery. The numeromancers and numerologists of the Thunderbolt Circle have always looked back to the time of the Gnome and his infamous 1954 thesis on 'How to Look Rich by the Standards of the World'.
Let me put it to you in simpler terms. It's not how much you own, or how much you have, or how much you can move, that necessarily makes you rich. It's how 'rich' is defined. So if you want your country to look rich, you ask around and see how people define 'rich'. Then you rig the outputs (not in the sense of 'bias the game', but more in the sense of 'setting the sails') so that the nation (or the game, or the ship) appears to be heading towards the right direction as far as 'rich' is defined.
This prestidigitation (also known as 'extremely fast number-management process'), if successful, will give rise to the impeccable claim that you are rich by any official measure. Whether you are rich or not in reality does not matter, since all that matters is what the numbers are supposed to say; all else is ineffable, immeasurable, beyond the terms of the discussion.
Let me put it to you in more relevant (to some of us) terms. Let's say your bank balance is US$500,000. You put the card into the machine and ask for a statement of account, and that is what is shown. But how do you know how much that money is? How do you know if you are rich? What if, unbeknownst to you, you have slipped into some sort of timewarp, and US$1,000,000 is actually the cost of a loaf of bread? Well, you'd have to ruefully remind yourself that half a loaf is better than none.
It's all about numbers, to some people.
But let's think about what happens when an education system begins to experience this sort of numerical subversion (or subvention, as some inventive people have called it).
In order to establish how good an education system you have, you decide that only quantitative measures are used. Even when you bring qualitative measures in, you use numerical rubrics and a final score expressed as some number. Worse, you require trend lines for everything, including the development of human virtue or social behaviour (for example, what percentage of the student population expresses confidence in the long-term prospects of your nation).
You decide to create an examination system that is vulnerable to the same prestidigitation. Everyone gets a number. The smaller, the number, the closer to Number One you are. The minimum magic number is 6. But if you are clever, this becomes 4, or 2, or zero. Or perhaps you make it the other way round; the higher the number, the more you've scored. If you get 45, that's the highest; if you get anything approaching 24 and below, time to start looking at the fine print.
The problem is of course that while numbers are good things to have in analysis, a lot of their relevance depends on two things: the assurance that the system is not being gamed in such a way that it fails to measure the quality of education (validity), and the assurance that everyone who does the same thing gets the same mark (reliability). If you look at these two things, you will realise that gaming the system — by doing things which are not educational but are number-manipulating or system-gaming — can give you marvelous results.
It's a bit like parents who insist on helping their kids with homework projects which are graded. You're not measuring the child's ability, but something else (the ability of the child to con the parents into doing it, the pressure society exerts on parents to do unethical things, etc). And if you build the whole edifice on it, the edification is not real.
Let me put it to you in simpler terms. It's not how much you own, or how much you have, or how much you can move, that necessarily makes you rich. It's how 'rich' is defined. So if you want your country to look rich, you ask around and see how people define 'rich'. Then you rig the outputs (not in the sense of 'bias the game', but more in the sense of 'setting the sails') so that the nation (or the game, or the ship) appears to be heading towards the right direction as far as 'rich' is defined.
This prestidigitation (also known as 'extremely fast number-management process'), if successful, will give rise to the impeccable claim that you are rich by any official measure. Whether you are rich or not in reality does not matter, since all that matters is what the numbers are supposed to say; all else is ineffable, immeasurable, beyond the terms of the discussion.
Let me put it to you in more relevant (to some of us) terms. Let's say your bank balance is US$500,000. You put the card into the machine and ask for a statement of account, and that is what is shown. But how do you know how much that money is? How do you know if you are rich? What if, unbeknownst to you, you have slipped into some sort of timewarp, and US$1,000,000 is actually the cost of a loaf of bread? Well, you'd have to ruefully remind yourself that half a loaf is better than none.
It's all about numbers, to some people.
But let's think about what happens when an education system begins to experience this sort of numerical subversion (or subvention, as some inventive people have called it).
In order to establish how good an education system you have, you decide that only quantitative measures are used. Even when you bring qualitative measures in, you use numerical rubrics and a final score expressed as some number. Worse, you require trend lines for everything, including the development of human virtue or social behaviour (for example, what percentage of the student population expresses confidence in the long-term prospects of your nation).
You decide to create an examination system that is vulnerable to the same prestidigitation. Everyone gets a number. The smaller, the number, the closer to Number One you are. The minimum magic number is 6. But if you are clever, this becomes 4, or 2, or zero. Or perhaps you make it the other way round; the higher the number, the more you've scored. If you get 45, that's the highest; if you get anything approaching 24 and below, time to start looking at the fine print.
The problem is of course that while numbers are good things to have in analysis, a lot of their relevance depends on two things: the assurance that the system is not being gamed in such a way that it fails to measure the quality of education (validity), and the assurance that everyone who does the same thing gets the same mark (reliability). If you look at these two things, you will realise that gaming the system — by doing things which are not educational but are number-manipulating or system-gaming — can give you marvelous results.
It's a bit like parents who insist on helping their kids with homework projects which are graded. You're not measuring the child's ability, but something else (the ability of the child to con the parents into doing it, the pressure society exerts on parents to do unethical things, etc). And if you build the whole edifice on it, the edification is not real.
Labels: Education, Games, Numbers, Statistics
etched with acid at 2:28:00 pm
0 Comments:
Post a Comment
<< Home