Read on Twitter
Okay, here’s some musing on the UK’s GCSEs and A-levels debacle.
Every person involved in data analysis should be watching this. It will become a case study in the future.
Every person involved in data analysis should be watching this. It will become a case study in the future.
The backstory: with exams canceled due to COVID19, it was decided in the UK that kids would have their grades assigned by their teachers’ estimates.
Bear in mind that these grades are what determine university placement for the kids.
Bear in mind that these grades are what determine university placement for the kids.
In the UK, the grades and university acceptance letters go out on the same day; it’s a country-wide thing. Kids send in their first, second, third etc choices for where they want to go to uni and their grades determine which one they get. It’s all centrally managed.
Nothing else matters except those grades. UK kids aren’t writing long essays trying to pitch themselves to individual colleges.
Ireland does the same thing except with a points system. Works well overall IMO.
Ireland does the same thing except with a points system. Works well overall IMO.
Anyway: so the kids’ grades were sent in, and you’d think that’d be the end of it.
But the UK have their heads up their own arses and of course they couldn’t leave it alone.
But the UK have their heads up their own arses and of course they couldn’t leave it alone.
They took all the grades and ran them through an algorithm that normalized them based on the performance of the school overall in previous years.
Schools that have done badly overall caused grades to drop.
Schools that have done well overall caused grades to go up.
Schools that have done badly overall caused grades to drop.
Schools that have done well overall caused grades to go up.
They did this to counter-act teachers’ bias.
Yeah, you read that right.
(Observe any data analyst’s head exploding at this point)
Yeah, you read that right.
(Observe any data analyst’s head exploding at this point)
So - the net effect of this absolute FUCKERY was that disadvantaged kids in lower income schools who had worked hard to do well and better their situation by getting into a good college were screwed. Their grades tanked.
Rich kids in much better schools saw their bad grades improved.
The teachers’ bias, if any, was replaced by class prejudice, and it was magnified.
The teachers’ bias, if any, was replaced by class prejudice, and it was magnified.
There was a massive shitstorm over this and it’s all been reversed and the algorithm scrapped. But the damage is done. University places have already been assigned.
The kids who got screwed now have to deal with possibly deferring, or taking exams again somehow.
The kids who got screwed now have to deal with possibly deferring, or taking exams again somehow.
And the universities are extra screwed - they only have so many places in each course. They have to possibly increase course places, with the costs associated with that, or tell a LOT of students that even though they got the grades, they can’t get in.
The rich kids who had their grades increased? They will not have them reduced. They still get to go to uni they don’t deserve.
The UK government just threw a whole cohort of young UK school leavers under a bus and let their rich counterparts off scott free.
The UK government just threw a whole cohort of young UK school leavers under a bus and let their rich counterparts off scott free.
So: the entire reason why they did this godawful algorithm shit is because the UK gov are VICIOUSLY classist and couldn’t bear the thought of someone undeserving from the wrong school getting into a uni course with all the deserving kids from higher income brackets.
IMO and I think I’m not far off the mark. There is no other good explanation for why they didn’t just trust the teachers.
The teachers know the kids FFS. You can argue that some might have given bad grades because of personal issues with a student...
The teachers know the kids FFS. You can argue that some might have given bad grades because of personal issues with a student...
But that would have been small scale, deal on a case by case basis.
This wholesale, across the board bullshit was clearly, CLEARLY going to be an absolute mess.
This wholesale, across the board bullshit was clearly, CLEARLY going to be an absolute mess.
Like I can’t even describe the level of utter contempt I have for whoever decided to make this happen because I cannot fathom the level of total INCOMPETENCE required.
I could have told you it would be a disaster and I’m not even British.
I could have told you it would be a disaster and I’m not even British.
I don’t know if anyone even looked at the results or did any validation on the output.
So why will this become a case study?
So why will this become a case study?
Well: this is a perfect example of garbage in, garbage out.
The reference data used to control the algorithm was accurate, but it should NEVER have been used for this purpose because it was highly biased.
The reference data used to control the algorithm was accurate, but it should NEVER have been used for this purpose because it was highly biased.
The data reflected class inequalities, and the people handling it should have KNOWN that from the start.
They either assumed that it didn’t, and school performance was independent of the students’ relative income levels, or they didn’t care.
They either assumed that it didn’t, and school performance was independent of the students’ relative income levels, or they didn’t care.
I’ll leave you to decide which it is.
Second: the use of the data in the algorithm obviously magnified class inequalities and unjustly screwed kids who need more help, not less. On a macro social level, that is a terrible idea.
Second: the use of the data in the algorithm obviously magnified class inequalities and unjustly screwed kids who need more help, not less. On a macro social level, that is a terrible idea.
Rich kids will do fine regardless. Poor kids need more help. Kicking them down instead is a good way to lower their contribution to society in future, and cost more in terms of public services.
The UK just kicked thousands of poor kids. Who knows if they’ll ever recover?
The UK just kicked thousands of poor kids. Who knows if they’ll ever recover?
Third: this is a prime example of data being improperly used to hurt vulnerable people.
This is a lot more common than you think. See predictive policing. (Do not trust cops under any circumstances)
This is a lot more common than you think. See predictive policing. (Do not trust cops under any circumstances)
Fourth: we desperately need more marginalized people in data analysis because I honestly think that would have raised the chances of someone saying “the FUCK is WRONG with you idiots” and putting the brakes on this bullshit.
So in short: this was a terrible idea from the start, enacted by HIGHLY STUPID PEOPLE, and I hope every other country is taking careful notes so that they won’t make the same mistakes.
My data analyst sensibilities are so offended by it all I can barely drink my coffee.
My data analyst sensibilities are so offended by it all I can barely drink my coffee.
