Accession Plan

I was idly looking through the access statistics for my Master's thesis when I realised that a third of the hits were from China, a quarter from Singapore, a quarter from the US — and the balance were from Japan and the Russian Federation. The four cities with the most hits were Beijing, Singapore, Palo Alto (CA, I presume) and Raleigh (NC, I presume).

I wonder a lot about the last two in each case. How odd. Why would anyone want a look at "A Study of the Conditions that Facilitate Servitor Conditioning in Atlantean Temples"?

Streaming

The natural tendency of both geography and biography is to stream things, to let nature take its course as diverse forces and pressures (forces confined, or spread out over areas) act on fluids. And so it is with human education.

The most effective thing is to let fluids find their level, and then channel them accordingly. The least effective thing is to do co-current heat transfer or to force all fluids to spread evenly in a thin and unnatural film over a thin and unnatural landscape (like some movies I've seen).

When people are streamed (at airport gates, by means testing, or in education), it is always a nett gain to society in terms of resources saved, as long as the streaming medium is relatively responsive to changes in the environment. What we shouldn't do is abandon any idea of streaming in the name of that soul-deadening anti-scientific anti-human thing called 'artificial equality'.

And that leads me to banding in school systems. If we no longer have it, is that a good thing? The point is that it is bad to avoid banding. This is because people do it anyway. It is biologically impossible for humans to avoid comparisons.

The better solution is to make ALL the data available and let humans test themselves against the data, making their own sense. Is a school that claims 'green' efficiency really so if it spends $500/head on electricity and water? Is it better to send your children to a school that is religious and cheap or secular and expensive? And what if one masquerades as the other?

What I propose is that far from scrapping banding, ranking and streaming, we should make all the data publicly available while suppressing the names of individuals. Show at least the bulk population data. For every school, release their accounts to the public for reasons of accountability. Let everyone see what schools do, so that people can make conscious pseudo-rational decisions that they can no longer blame on anyone else.

How's that for a different take? *grin*

Competition Curve

Look at it, the beautiful idea of the normal curve. For that is what it is, it is a mathematical construct which smiles at us like a lazy anaconda and tells us, "This, this is normal, this is what things should be. BURP."

And the curious thing about this normal curve is that it is flattish at both ends and it flattens out in the middle, but there are two spots at which the degree of curvature is very high. One of them is when the tail end, ignored, reviled and despised, attempts to climb up to 'normality'. The other is when the middle, bulging with half-digested ideation, attempts to squeeze its way into the thin head of the engorged snake.

The problem is not with the former. There is so much more space in the middle that we can accommodate people drifting towards the middle. But here lies mediocrity, and while those below the middle wouldn't mind entering this zone, those in the middle are aching to get out of it.

They will push up and on, they will strive to be the best, they will seek a better age. And therein lies the rub: it takes proportionately far less resources to get the under-average into the average than the average into the excellent. It is extremely painful, in fact.

And that is why the complaints about how tough any system is will not come from the bottom, but from somewhere just below the neck.

Normal is an Island

The problem with humans is that they are limited by the presence of a relatively small corpus callosum between the hemispheres of their brains. This makes them prone to reification, or at the very least, oversimplification. That's why they like universal laws, universal beliefs, universal rights; and if not, at least the appearance of such things through the application of normative fiat or normal distributions.

If a fundamental assertion of human thinking is that a less complicated answer is always better if it explains exactly the same things as well as a more complicated ones, then what if the things are not always exactly the same? This is true with comparisons between island-states, city-states, nation-states, federated states, overlarge conglomerated states, and so on.

I mean, why on Earth would citizens of one of the 50 (plus change) United States want to compare themselves with the last major (but very tiny) city-state in the world? Or with the ancient Confederatio Helvetica? Or anywhere else? And the reverse applies. What is world-class, but the idea that the entire world rates you in the same way, which is silly in a world of differences.

But surely there are absolute morals in a human world? Absolute rights, absolute ethics? No, I'm afraid not. There cannot be such in any biological population, since you can always engineer the environment or the circumstances to allow for any deviation from the defined norm. Worse, 'normal' in this context, and its supposed antonym 'deviant', don't mean what you think they mean. They are statistical ideas — the perpendicular to the turning point of a distribution is the 'normal', the spread of difference from that norm is 'deviation'.

I am certain there are absolutes. I just don't think that they can be derived from statistics, which by their nature are relativist.

The Five Percent Rule

It has long been heuristically suspected that in any population, only about 1 in 20 will actually achieve a performance that is at a level of 1 in 100 or better (i.e. better than 99% of the population's performances). This kind of supposition is not always true of all populations, but it is a good heuristic.

Look around you. Who is the best person in your class, group, set or other population for a given task? Is that person capable of beating 99% of all performances by your population? That person must be at the very worst a 1-in-20 phenomenon — in a group of 40, only 2 such are expected.

In a staff of say 300 people, this would mean not more than 15 are A-class. By this I mean using a measure of something like: A = top 1%, B = next 4-5%, C = next 10-15% and D = next 25% or so. Anyone after the top 45-50% should be considered a failure.

Why? Because in such populations, the people have already been selected for the job. If they can't do the job well, the question that should be asked is, "Why are they so bad?"

And this is one of the reasons that I believe all human resource executives and suchlike should be fired. They appear to serve no useful purpose at all and I am willing to think that basic filters plus luck-of-the-draw would do better.

The Bloat

Wolff has found himself back in the colours. With blade and long lance, on fully-caparisoned horse that is not a horse, he sits on a hilltop and gazes into the distance.

And there it is, as Chesterton puts it, 'like a fungus of a leprous white and grey'. It is the Bloat. Wolff remembers when a squad had one leader, and several squads a commander. The Romans had squads of ten men.

But at the Citadel these days are 265 who have been given rank, and of these, 90 have a higher rank. 90 out of 265. It is more than a third. It is as if out of every three, one is a leader.

Wolff shakes his head. He has looked at the banners of the Green Order, and for all their prayer and service, they have only one in four. It is the same elsewhere, perhaps as low as one in five or one in six. But at the Citadel, one in three.

He is reminded of the mighty Parkinson, who pointed out how the Admiralty of old swelled at the top even as the ships became smaller in size and the men became fewer in number. He too was based in Atlantis for a time and knew whereof he spake.

He remembers when the citadel was 1200 soldiers, with 80 knights, and among those, 20 of higher rank, who shared the work and tasks appointed unto all. Now it is 3000 soldiers, with 265 knights amongst which are 90 of higher rank.

How did this perversion come to be? It was all the fault of the Grand Inquisitor, Wolff decides, he who carefully scattered largesse around to secure his place, but not the Order. And hence the general incompetence, for if one needs not prowess at arms to win the prize, why should one aim for prowess?

90 in 265. Wolff laughs. The winnowing will not be a happy one.

=====

Note: The fictional adventures of Sir Wolff do provide much that is of interest. You can find them linked here. Just ignore the one about earwax. That was an accident.

Statistical Review

I was just reviewing my data about the Citadel and its performance. Wolff looked at my tablet, and laughed. There was something not quite pleasant about his laughter. In fact, it was quite pointed. So I looked at the page more closely.

In that document I saw these figures for 2009:

Table 3 above shows that our students’ grade average has increased over the previous year — 6.48% as compared to 6.47% in 2008 in spite of a larger cohort of candidates.

And so, like any conscientious researcher, I decided to see if this was significant.

First, I found an obvious and rather sloppy error. 6.48 and 6.47 were absolute averages, not percentages. The 6.48 was an average score from n=425 and the 6.47 was an average score from n=406.

Second, I found that this was not a significant increase. An increase of 0.01 over 425 students is +4.25 points — quite possibly the effect of just one student doing a lot better than his counterpart in the previous cohort.

Actually, let's suppose that every student did better by half a grade point (or 0.083 per subject) on average. Then you'd have an average score of 6.55 at least. That this did not happen tells me that the Citadel has reached a sort of plateau. It had a bad year in 2008, but 2007 and 2009 seem about the same.

Ah well. Time will tell. With a cohort of about 450 this year, we should see an average of 6.56 for a significant gain. I suspect this will not happen. I could be wrong.

=====

Note: Yes, I am aware that students with a full 42 subject points (over 6 subjects) can't do better than 7.00. However, one should hope that those doing badly will improve sufficiently to make up for lack of improvement at the top, since it is easier (percentage-wise) to improve a poor grade than a good one.

Fear of Aliens

Renowned physicist Stephen Hawking recently expressed the idea that one ought to fear aliens. Here is the key passage in the Times article on his Discovery Channel show:

He suggests that aliens might simply raid Earth for its resources and then move on: "We only have to look at ourselves to see how intelligent life might develop into something we wouldn’t want to meet. I imagine they might exist in massive ships, having used up all the resources from their home planet. Such advanced aliens would perhaps become nomads, looking to conquer and colonise whatever planets they can reach."

He concludes that trying to make contact with alien races is "a little too risky". He said: "If aliens ever visit us, I think the outcome would be much as when Christopher Columbus first landed in America, which didn’t turn out very well for the Native Americans."

The fallacy in this otherwise brilliant man's idea is clear. "We only have to look at ourselves to see how intelligent life might develop..." assumes that alien life is not alien. It is of course rational to base our concepts on how intelligent life MIGHT behave on the only clear example we have, the one on which we base all our concepts of intelligence — humanity. But to then say that aliens might develop like us ignores the very mathematical brain he cites earlier:

"To my mathematical brain, the numbers alone make thinking about aliens perfectly rational," he said. "The real challenge is to work out what aliens might actually be like."

The numbers alone should make thinking about aliens also factor in the likelihood that aliens are extremely unlikely to be anything like us. But there are also a few other statistics that should be of importance: for example, given that we have encountered no recognizable trace of alien life so far, inductive reasoning should hold that there is no such thing as alien life in a form that we can recognize as such.

It is similar to the statistical idea here: the longer we exist, the more we are certain that a) humans are the only humanly-defined human-level intelligence around, and b) life only exists on our planet. How can that be? Surely the laws of probability must dictate that there is other life elsewhere?

Actually, here we run aground on the rocks of infinity, so to speak. In theory, anything is possible in an infinite universe. However, if there are any universal laws at all, this constrains certain events from happening; and empirically speaking, the 'certitude' that anything is possible does not mean that any specific thing has indeed occurred. It is unthinkable to some scientists that alien life does not exist, but that is the triumph of imagination over intellect (Einstein's definition of love, actually).

Here's another point to ponder: in an infinite universe, there can be an infinite number of intelligent alien races — but in an infinite universe, the 'quarantine distance' between them might be too great to overcome. Perhaps they are all out there, but too far away, like individual colonies of sulfur-loving bacteria near ocean vents, that will never ever meet members of other colonies.

I don't fear aliens. But I love cats, and fear sharks. I am neutral towards dogs in general, and dislike many kinds of insects. The chances that these biases will affect my future transactions with (for now) imaginary aliens are statistically nil in an infinite universe. Think about what that might mean.

Indirect Thinking

Let's take a look at a hypothetical case, the subject of a thesis in the field of education. Supposing you have a school whose results are among the best in the world, statistically speaking. To what might you attribute the cause of this happy state?

Asking this about the results is somewhat similar to evaluation of any product. The logical causes are the management, manpower, materials, machines, or methods. There might be some overlap, but these generally fall into separate categories.

Then five kinds of research problems might be said to arise, one for each of these putative causes. You need to see if any of these is sufficiently and provably superior to the equivalent somewhere else in order to possibly attribute good results to it.

In the case of my research, extensive brain-contorting (and sometimes brain-convulsing) work has shown that I can only prove the case for one of these categories.

And yet, in the general media and in our general way of thinking, we tend to attribute success to the other four categories more than we attribute success to this one. Why?

Perhaps, the problem with schools is the self-fulfilling nature of education. It's easy to show that superior students produce superior results. What we'd like to know is if they are superior in any other ways besides previous results, and how they got that way. Are they smarter, do they drink more coffee, have they got richer parents, are they motivated by fear (or pride, or greed, or any other one of a thousand human motivations)?

When a school spends huge amounts of money on staff, structures, and administrators, how can it be shown that this has any impact on the final result? And yet, people assume that it has.

In some cases, this is easier to show: a science lab or a dance studio are strongly facilitating aids to improvement in their respective disciplines. In some cases, this is harder to show: can a 35-year-old ex-government policy wonk do better than a grizzled 67-year-old matriarch as a school leader? Do oak panels depicting past triumphs and famous alumni encourage modern students to do better? Or is all of this simply reducible to the ancient axiom, 'Money Breeds Success'?

It would be kind of sad if I had to reduce my answers to the two ideas that a) people with good previous results tend to produce good future results in the same general area, and b) money breeds success. I would have done no better than many previous generations of researchers, then.

But hold on, you might say. How about all the examples that buck the trend? The poor minority kid who becomes valedictorian, the uncouth gangster who becomes a leader, the ramshackle makeshift hut that becomes a centre of great learning? What about the rich kids that flunk out, the expensive schools that flop?

I'd have to say, "Exactly." They buck the trend, they are newsworthy precisely because they are some distance away from the norm. The statistics don't lie. Live in a rich estate, go to an expensive school, you'll do better academically. You'll do better in life too.

Yet, there is one small factor that gives hope to the rest of us. It's sometimes the case that a person from a less-advantaged background can show superlative success because of a rare confluence of factors: seizing a rare opportunity, showing great personal drive, having enough coffee, and so on. This success may have vastly disproportionate effect across time, thus shifting things enough for society to change somewhat.

It is this sort of success story that partly drives the altruist in us. The human altruist (teacher, doctor, pastor) doesn't mind providing services to anyone. But somehow, it's a lot more satisfying when someone makes maximal use of such services to do far more than you'd expect from the statistics. I don't mind teaching smart kids who assume they are smart all day — it's great if they learn to be smarter or if they learn that 'smart' is something that they were taking too much for granted. It's even better, though, when people who kept telling themselves they weren't smart suddenly realise that they are smarter, and can be even smarter. (Yes, however you want to define 'smart'. What I mean is 'good at something'.)

And it's best when the norms are shifted so that there is a broader range of successes. Which leads me to one last point about schools. A school should be able to take in anybody and help that person become a better person. What a school shouldn't be doing (unless compelled to by higher authority) is to deliberately deny students who look like they are going to be harder to teach. Without these students, how on earth are teachers going to learn? Heh.

Justifiable Assumptions and their Consequences

I'm still ploughing through the hefty tomes that constitute the Gnome's wise sayings from his apical position at the top of the Atlantean hierarchy from 1959 to at least 1984. He makes an awful lot of assumptions, and sometimes, you have to take into account his weaknesses as well as his colossal strengths.

For example, the Thunderer once said that the Gnome was a bad judge of character, able to spot talent but not able to sense the constitutional underpinnings of it within each individual. From various readings, you can probably guess that the Gnome assumed something like a 1 in 3000 success rate at finding world-class candidates for the Atlantean High Command. The Thunderer's pragmatic assessment would reduce that to perhaps 1 in 10,000.

The Atlantean education system currently contains a shade under 30,000 educators up to the high school level. Of these, about 800 are principalities or deputy principalities. This means that if we are to take the founding fathers of modern Atlantis seriously as to the level of available talent, there are not likely to be more than three world-class educators in the whole lot.

The commonly-held assumption is that the best talent is actually sucked into the Administrative Service, the priesthood which controls all the highest functions of Atlantean society. Approximately 300 officers occupy about 600 posts; a large number of these officers run the commercial arm of Atlantean government.

The current Atlantean population is about 5,000,000 souls. If 1 in 10,000 is a candidate for the high Command, then there are only 500 people who can run the show. 300 of them are in the Administrative Service; that leaves 200 for everyone else.

Of these 200, we assume that 3 are somewhere in the education service, swamped by about 29,000 other officers of variable but lesser abilities. Yet somehow, we have managed to cultivate a 'world-class education system'.

My conclusion at this point is that if this education system is 'world-class' (whatever that means), it's a triumph of systems engineering over human leadership. It must be a system that works regardless of the manpower available. In fact, as I've published elsewhere, it is a system which has extremely fluid manpower characteristics but extremely rigid structural characteristics.

It is like a system of concrete channels within which seawater and all kinds of marine life circulate. The channels are fixed, but what kind of 'catch of the day' you get depends a lot on luck — the quality of your fish is not guaranteed, except that it is the kind which will thrive in such a system.

If I were less charitable, or willfully cynical, I'd call it a goldfish bowl system. But there is one key difference: with a goldfish bowl, the fish see each other and the observer can see them all — but in this system, the fish do not all see each other, and the observer can't see them all either. It's concrete and steel, not glass and light.

I can't wait to get to my final conclusion.

Value-Addedness

It struck me the other day, as I was reviewing the statistics on school performance, that something was going rather wrong with certain calculations. Let me explain.

Here is how we calculate something called value-addedness in a nationwide high-stakes standardized examination:

1. You get everyone in Grade 6 to be tested in Math, Science, English and a second language. This gives you a score for the entire country.
   
2. Then you test them again at Grade 10 and Grade 12 (well some skip the Grade 10 test).
   
3. This gives you a statistical measure of what kind of performance at Grade 10 or Grade 12 students ought to produce, relative to their score at Grade 6.
   
4. If a student has got a significantly higher score than this expectation, this is called 'value-addedness'.

For example, if Jack scores 250/300 at Grade 6, he is in the top 10% and is expected to get straight As at Grade 10, and so on. The method is a bit more well-refined than that, and a little worse. For a start, they don't actually factor how well Jack did in individual subjects at Grade 6, but they do track how well Jack's seniors did in individual subjects at Grades 10 and 12. They use a rolling average over n years to computed the expected score, which is only useful if there is no clear upward or downward trend.

If the trend is upward, then the rolling average is actually lower than the current expectation; if the trend is downward, then the rolling average is actually higher. If a school policy means that everyone must take subject X, the chances are that X will regress towards a mean; if a school policy means that only the elite can take subject Y, the chances are that Y will have so high a mean score that it becomes meaningless to think about what value-addedness means.

What if, however, the results for the whole country are said to be better than before, every year? Well, then that's very odd. It means that value-addedness should be declining. Actually, if the statistic is distributed normally, then half the population should be above the mean and half below. If the country is doing better AND the value-addedness is rising, then there must be a very long tail or something.

But the interesting thing is, what if the entire national population is in the top 50% of the world at Grade 6 (or 10, or 12), and it isn't anymore after Grade 12? Does that mean the system is screwy (or screwed)? Something to think about...

Pandemic (Redux)

Yes, yes, I know I have talked about disasters and pandemics and all that sort of thing way too much. From now on, if you haven't already been doing so, you can keep track of the 2009 influenza 'they said it would not be one' pandemic on your own at this reliable and frequently updated webpage.

Atlantis has about 600 casualties as of this hour. Someone should do a per capita analysis. I think, that as Jared Diamond has pointed out before, city living is a prime cause of the spread of contagion. Cities are reservoirs for this sort of thing, since people have little or no choice but to come into range of the diseased. If your entire country is a city-state, then you can expect to be pretty much doomed to suffer the complexities of pandemic management.

Meanwhile, the Southern Cross tribe has claimed that their own high figures are due to better detection of disease, and that if everyone else (especially the people of the Beautiful Land) were as honest as they, the figures would be much higher. Someone said that about a million of the Beautiful People have got it, although they say they only have about 25,000.

Currently, things aren't too bad. But way back in 1918, things got really bad. An estimated 3-6% of the world's population died then, from the 1918 H1N1 influenza virus. Almost a century later, could the bad old times be set to roll once more?

Meanwhile, the Indic Syndics have claimed that their billions of people have only suffered about 70 casualties and no deaths. Statistically interesting, I'd say. The 1918 bug killed 17 million of them.

Asymmetrical

The concept of asymmetry is an interesting one, for one would expect it to be the natural state of a random universe. At the same time, the idea of randomness is that it should eventually be chaotic all round, and hence oddly symmetrical. The two concepts have always been at odds, sort of.

Here's an illustration. Throw 20 coins in the air and let them land randomly. Odds are that you will get 10 heads and 10 tails, right? Actually, not quite. The odds are higher that you will get a 9-11 split; that is, the combined odds of getting 9 heads and 11 tails or 9 tails and 11 heads are higher than that of the 10-10 split.

It's this phenomenon that bridge players exploit when thinking about who has what cards. If you have five cards in a suit and your partner has none, then it's more likely that of the eight remaining cards, your opponents have a 5-3 split than a 4-4 split. That dictates your pattern of play: they are more likely to run out in 3 rounds than in 4.

The universe, paradoxically, is like that. The average is like a 10-10 split, but the reality is normally a 9-11 split out of 20 coins.

=====

Meanwhile, elsewhere, THIS is a brilliant idea. Jane Austen and the undead, all in one. Wow, what a classic!

Gaming the System: A Brief Introduction

The rise of Atlantis has always been predicated on quantitative sorcery. The numeromancers and numerologists of the Thunderbolt Circle have always looked back to the time of the Gnome and his infamous 1954 thesis on 'How to Look Rich by the Standards of the World'.

Let me put it to you in simpler terms. It's not how much you own, or how much you have, or how much you can move, that necessarily makes you rich. It's how 'rich' is defined. So if you want your country to look rich, you ask around and see how people define 'rich'. Then you rig the outputs (not in the sense of 'bias the game', but more in the sense of 'setting the sails') so that the nation (or the game, or the ship) appears to be heading towards the right direction as far as 'rich' is defined.

This prestidigitation (also known as 'extremely fast number-management process'), if successful, will give rise to the impeccable claim that you are rich by any official measure. Whether you are rich or not in reality does not matter, since all that matters is what the numbers are supposed to say; all else is ineffable, immeasurable, beyond the terms of the discussion.

Let me put it to you in more relevant (to some of us) terms. Let's say your bank balance is US$500,000. You put the card into the machine and ask for a statement of account, and that is what is shown. But how do you know how much that money is? How do you know if you are rich? What if, unbeknownst to you, you have slipped into some sort of timewarp, and US$1,000,000 is actually the cost of a loaf of bread? Well, you'd have to ruefully remind yourself that half a loaf is better than none.

It's all about numbers, to some people.

But let's think about what happens when an education system begins to experience this sort of numerical subversion (or subvention, as some inventive people have called it).

In order to establish how good an education system you have, you decide that only quantitative measures are used. Even when you bring qualitative measures in, you use numerical rubrics and a final score expressed as some number. Worse, you require trend lines for everything, including the development of human virtue or social behaviour (for example, what percentage of the student population expresses confidence in the long-term prospects of your nation).

You decide to create an examination system that is vulnerable to the same prestidigitation. Everyone gets a number. The smaller, the number, the closer to Number One you are. The minimum magic number is 6. But if you are clever, this becomes 4, or 2, or zero. Or perhaps you make it the other way round; the higher the number, the more you've scored. If you get 45, that's the highest; if you get anything approaching 24 and below, time to start looking at the fine print.

The problem is of course that while numbers are good things to have in analysis, a lot of their relevance depends on two things: the assurance that the system is not being gamed in such a way that it fails to measure the quality of education (validity), and the assurance that everyone who does the same thing gets the same mark (reliability). If you look at these two things, you will realise that gaming the system — by doing things which are not educational but are number-manipulating or system-gaming — can give you marvelous results.

It's a bit like parents who insist on helping their kids with homework projects which are graded. You're not measuring the child's ability, but something else (the ability of the child to con the parents into doing it, the pressure society exerts on parents to do unethical things, etc). And if you build the whole edifice on it, the edification is not real.

The Analysis That Should Not Exist (Part II)

This should probably be called 'The Analysis That Will Not Exist'. It is interesting to compare the information released by the best schools in the world about their performance in the International Baccalaureate Diploma examinations. It is even more interesting to see what information is no longer being released.

On one hand, the best IB school in the UK has released its usual simple statistical bulletin. There is no interpretation; only the facts of the case are presented, and it is clear that they have done well. The source PDF can be found here and was linked in my previous post. It is easy to see where the school has improved, and that over the years, the school has done well indeed. The Times Educational Supplement listed this school as the best independent school in the country, and it is clear why.

On the other hand, the best IB school in Singapore has released its usual report. There is some interpretation of the facts, it is clear that they have done superlatively well... but hey, the subject statistics are missing. I wonder if this has anything to do with my previous post(s). You can compare the 2008 results (PDF here) with the 2007 results (PDF here) and see how the data presentation has changed since the last time. In aggregate, it is clear that this school has improved; it is not so clear how this change has come about (although my last post has a theory which I don't think they want me to test).

One thing is for sure. I stand by my contention that the first school probably has far better results in English HL—it is hard to beat a 90% distinction rate. They have a history of it, and statistically, I believe at a 99% confidence level that I am right. It is to be expected. Maybe, they did better in History and Economics as well.

Similarly, by analysing last year's results, I expect that the second school did far better in Physics, Chemistry and Mathematics HL, and that a far larger proportion of the student body 'majored' in these disciplines (i.e., took them at Higher Level rather than Standard Level). They probably did better in Business & Management too.

These are not contentions which slight either school or are designed to insult or deride the efforts of the hard-working teachers and students. Rather, they are efforts to evaluate school strengths as they vary across the world. If my theory is correct, the Singapore Science/Math effect shown by TIMSS is a product of the local educational culture and affects the results even in other kinds of testing.

I am sure that both schools will do even better over time (unlikely though it might seem, since they have already done so well). I congratulate both schools (although I don't think I have the standing for either to take notice) on a job well done, and I do indeed believe that the best is yet to be!

The Analysis That Should Not Exist (Part I)

There is an insatiable curiosity in educational researchers that leads them to defy social norms and say the darndest things about the societies in which they live and the societies in which other people live. No doubt, this is a pronounced tendency of researchers in other disciplines in the arts, humanities and social sciences.

However, education is different; like history, it is a discipline that almost every human partakes in from birth. It is a discipline whose right of practice is almost certainly very close to being a basic human right. This is not so for many other disciplines, and in fact, if it were not for education, there would not be many other disciplines.

I have said all this to explain why educational researchers tend to strike more at our core beliefs (and with less accuracy) than most other kinds of researchers. Having said this, I am going to commit myself as one of that reckless crowd and say some things about the International Baccalaureate Diploma Programme (IBDP) and the city-state of Singapore.

Here are some basics. The IBDP is a pre-university (or senior high school) educational programme. Full details can be found at the IBO website. It is progressive, has many things going for it, and is an ideal education system (well, at least, it is one based on ideals which are clear and widely disseminated). There are some detractors, with interesting arguments linked to malignant globalism and suchlike, but the Programme is generally taken to be A Good Thing (as far as the Asia-Pacific region is concerned, at least, and well beyond).

The main curricular structure of the Programme (besides its ideals and worldwide goals) revolves around three core disciplines. These are Theory of Knowledge (TOK), the Extended Essay (EE) and Creativity, Action, Service (CAS). There are six academic groups which can be summarised briefly thus: Literature, Language, Humanities, Sciences, Mathematics, Aesthetics. Students must take a discipline from each group (although the last group can be replaced by another of the first five groups). I've written more about this in a previous post.

Singapore is known to do well in Mathematics and Sciences. This is a key finding that has been repeated many times; the Third International Mathematics & Science Survey (TIMSS) has provided evidence that this is so (and yes, it has been much proclaimed in PR pieces like this). My conclusion would be that a Singapore school that takes in average Singapore students should outperform (on average) most schools overseas in the IBDP groups 4 and 5 (Sciences and Mathematics). Perhaps more accurately, if the students from Singapore's top 5% are matched against the students from anyone else's top 5%, then the Singaporeans should outscore them in standardised testing.

This leads to a predictable situation. A majority of Singapore students taking the IBDP would take two Science subjects and one Mathematics subject; having done so, the quality of the average grade in these areas would outweigh any deficits in any other areas. Singapore students should do very well then, but only if they concentrate on scoring in the sciences and mathematics. I am not so sure they would do as well in the humanities, literatures and languages which the IBDP offers.

Again, this prediction is made based on very general statistics. However, it is possible to pit the best independent IB school in the UK against the best independent IB school in Singapore. The head-to-head analysis looked interesting, so I carried it out. Guess what I found...

Making The Pond Bigger

I was a teenager not that long ago. Well, not that long ago. By that time, John McCain had entered the Senate. At that time, the internet was non-existent except in the highest levels of a small group of nodes. It was possible to look around me and take comfort in the false belief that I was really pretty smart compared to many people. Maybe even smarter than half the people around me, which would have made me one tiny spot above average.

The thing about statistics is that they are not easily mapped into properly meaningful ideas in the human brain. I use that word 'properly' for a reason – it is all too easy for them to be mapped into meaningful ideas with the wrong meaning. Take, for example, the last sentence in my first paragraph above. If I went around saying I was smarter than 50% of the population, I'd only be attributing slightly-above average intelligence to myself (however you measure that slippery concept of intelligence). But I'm sure it would be taken as some sort of elitist comment.

Let's be honest, though. I classify myself as 'pleasantly ugly' (i.e. a bit below average in looks), 'fairly intelligent' (i.e. at least about 1 standard deviation above the mean on most scales), 'normally fit' (i.e. within 1 standard deviation of the mean on most scales), and 'normally healthy' (i.e. within 1 standard deviation of the mean on most measurements of health).

All this self-assessment was not so easy to make when I was young. You only had the evidence of your immediate senses, and I had the disadvantage of going to a school in which lots of people seemed smarter and fitter than I did. As I grew up, however, I began to realise that the immediate population of my youth had been somewhat biased.

In other words, as the pond grew bigger, I realised that while there were amazingly huge numbers of people who were incredibly much smarter than I was, there were also perhaps more people who were not. This made me feel a lot better, although not that much better.

The point of all this really is that if you have spent your whole life in a small city-state, you should get out a bit. Look at as much of the whole pond as possible before you think of what kind of fish you are. This is increasingly important as access to the global pond opens up. This especially goes for people who are discontented with life after looking at their neighbours.

You know the kind: *grumble* *grumble* I only earn US$4000 a month and my monthly costs (after paying the bank and all the bills) are now US$3200 so I only save US$800 a month. My neighbour has US$16000 a month, so he can probably save US$12800 a month.

Well, that kind of pay probably puts you both in the top 2% of all wage earners in the world. In a small city-state, you can even determine very exact and accurate figures for what percentage of your pond-people earn more or less than you. What's scarier is that in this particular city-state which is the focus of my research into education reforms, it seems fair game to determine what the best primary schools are based on economic principles of supply and demand. That's not so bad, except that it might be true.

I guess the great thing about access to the whole pond via the Internet is that it's easier to find data about where you really are in life, if you need such things. I know a highly accomplished person who was #1 in everything, but ranked very poorly in the big pond; at the same time I knew one who was thought to be pretty low-ranked, but whose data when compared with the rest of the big pond made a big splash.

In the end, I suppose that the economically-minded fish will do the economically-minded rational thing and migrate to the parts of the pond with more food. Then they will grow bigger and fatter until the fishermen figure out where the big fish are.

Conclusion: somewhere between 1 and 2 standard deviations above the norm is fine; don't push your luck. Heh.

Crossing The Bar

Something very significant is going to happen as Sunday night gives way to Monday morning in a few days' time. However, it will only be noticed by fans of certain bar charts. A long-standing spike indicating high traffic will suddenly disappear, heralding the establishment of normality in at least one way. I just thought I'd mention it. Heh.

And a happy August to all, especially the August personages out there!

The Black Swan

Those of you who have heard me speak before might remember the mentions of black swans, chaos and catastrophe theory. This all came back to me as I read Nassim Nicholas Taleb's The Black Swan: The Impact of the Highly Improbable. It is an interesting book; it is, as far as I know, pretty ground-breaking in exposing the facts people don't often think about.

Taleb destroys, with great finesse and remorseless logic, the idea that the normal distribution is a useful predictor for the big movements in history and in life. In fact, statistical arguments depending on the normal distribution and its proxies seem to end up giving us more-of-the-same, mediocre possibilities. History may have these as her staple diet, but the muscles and bones of history require sterner stuff.

But this is not all of his argument. He goes after Platonicism and other models that seek to shoehorn data into ideal models. He comes out in favour of the legitimate imagination and the knowledge of unknowledge, this last typified by Umberto Eco's antilibrary. An antilibrary, as he deftly explains, is that portion of a person's library which has not been read and yet is an integral part of that library; it is thus the mobile resource on the intellectual battlefield. People who don't read are one thing; people who have nothing they want to read are quite another.

The problem is that the world tends to seek to find consoling and conforming narratives to explain chaos, catastrophe and crisis. St Paul's epistle to the Romans mentions as much: be not conformed to the world. I remember very distinctly my meditations on this the day before my career was elevated to a different plane. I only read Taleb's book yesterday. To my surprise, I found myself feeling validated.

I do believe firmly that vision must be broad and imaginative. I also believe firmly that contrasting and outlying views must be fully examined before the decision is taken to reject them. There is, as many have said before, a bias against those who solve problems that haven't occurred yet, or who invent problems which might never happen. This bias favours those who solve problems that have come to be; the problem about such problems is that once they have already arisen, they are much more expensive to deal with than when they were merely imaginative possibilities.

That is why we must be transformed by the renewal of our mind. That is why we must see with 'blinding sight', with inspiration and not with the conventional wisdom. That is why we must not conform to the normal distribution of the world. We are all, in potential, far better than that; it is mediocrity that creates the curve and treats the outliers and possibilities of excellence as fantastic and irresponsible. On the contrary, it is the mediocre mind which is being irresponsible, which holds back the flowering and blasts the stalk before a bloom unfolds. How sad it is, to see the garden burn.

Statistical Scores

I've been accumulating twenty years' worth of detailed statistics and statistical analysis. This comprehensive archive has been of great value to me in my work and beyond, as the facts are always useful for rebutting reckless allegations and falsehoods about how good or bad some programmes or programme directors have been.

The oddest thing is that I never really noticed how successful I had been until I was told I had been unsuccessful. It wasn't pride that drove me to analysis, but curiosity. Had I really been that bad? Then I had a good hard look at the data and was pleasantly surprised to see that I had presided over the most successful span of years in the history of that institution.

These are the facts: the year preceding my first year in office was the nadir of the entire operation, and my final year was the zenith. The lie had been told that institutional results under me had been on a downward trend. I have very good reasons to believe otherwise, and that we actually maintained a positive record against other institutions of like profile. The national data supporting this were assiduously compiled by me, and some people were just not interested at looking at it in all that time.

The thing is that the data are the data. Statistics may be used to obfuscate the facts, but there are some data which under any reasonable analysis look good, some which just look bad. Although it has been said that there are 'three kinds of lies: lies, damned lies and statistics', it is not necessarily implicit that statistics are more false than lies.

Statistics are still susceptible to careful analysis and have the possibility of truth; a lie is untruth by definition while statistics are only untruthful by implication, and only in this one quotation (for which, we apparently have Benjamin Disraeli and Sam Clemens, to thank). It is of course true that statistics can lie in the application and in the explication as well as the implication, but we as reasonable human beings can attempt to make them lie less or even tell a story that is true enough for honest use.

Twenty years' worth of data, all in my hands. Are these a score to settle scores? Of course not. I would not stoop so low, unless dragged down into the gutter by some of those damned lies. Instead, one might seek to soar instead, above the hurly-burly of the mud-slinging. Then again, as someone said to me, it is good to keep a sharp knife and a sharper wit.