The Chicken Pie Problem
Some 'real world' GCSE questions bear little relation to reality
06 November 2017
Last week a teacher emailed me a copy of a Higher Tier GCSE maths paper, and asked for my opinion. (It was the Edexcel November 2017 non-calculator paper, since you ask).
Now I don’t tackle GCSE maths questions very often. I’m rusty on some of the terminology (how do you do a box plot?), and I have to work from first principles if there’s a question about geometry or trigonometry, which slows me down.
But I do spend a lot of time on ‘problem solving’ and since that’s the new emphasis in GCSE maths, I reckoned I’d be pretty well equipped.
I decided the only fair way to form an opinion of the paper was to try to sit it in exam conditions. I cleared the table, switched off my phone, set up a timer and began.
As it turned out, it wasn’t quite 'exam conditions'. About halfway through I had to take a break to go upstairs and read bed-time stories, but apart from that it felt pretty real.
And I found it remarkably stressful. Particularly because, having breezed through question 1, I got in a tangle with question 2. It was an 'age' puzzle where for some bizarre reason they wanted me to work out not the ages of the three characters, but the ratio of ages. I decided to come back to it later. I hadn’t expected to struggle so early.
AND NOW - THE CHICKEN (AND STEAK) PIE PROBLEM
And then I got to Question 13, the only question in the paper that features probability. Probability is one of my strongest suits.
Here is the probability part of the question:
A factory makes 450 pies every day. The pies are chicken pies or steak pies. Each day Milo takes a sample of 15 pies to check. The proportion of the pies in his sample that are chicken is the same as the proportion of the pies made that day that are chicken.
On Tuesday, the number of steak pies Milo needs in his sample is 6, correct to the nearest whole number.
Milo takes at random a pie from the 450 pies made on Tuesday.
Work out the lower bound of the probability that the pie is a steak pie.
OK, you might need to read that again. I certainly did. Twice.
The question being asked was unusual. ‘Work out the lower bound of the probability…’. What does that even mean? Whatever happened to straight probability questions like "what's the chance there are two blue hats?". After a pause, I did figure out what the examiners were looking for. But I had another concern. WHY would anyone need to work out the probability that it was a chicken or a steak pie? Milo already knows this probability, because he knows how many pies he’s selected and what proportion of all the pies are chicken. This pseudo real-world maths question is a grotesque distortion of any statistical scenario that a production line tester is likely to face.
I wondered if it was just me that had an issue here, so I asked a leading statistician for their opinion on the chicken pie question. Their comment: ‘This is really contrived. This isn’t how sampling is done. And also, what is the “lower bound for the probability” - is this related to confidence intervals?’
Then I asked a friend who is a management consultant specialising in decision-maths and big data. He said: "I don't understand what they are asking for."
Wow. If adults at the top of their profession don’t understand the question, then what hope is there for the average teenager?
I don’t envy examiners. They have been instructed to make GCSE more of a challenge, to stop spoon-feeding pupils, and to set questions that link different parts of the curriculum. Many teenagers find it easier to grasp a problem if it is set in a world of people doing things with familiar objects, such as chicken pies, rather than abstract things like ‘x’.
Unfortunately, the outcome is often the worst of all worlds. You get a problem posing as if it’s the real world which is actually nothing like the real world.
And there’s something else that makes this different from the type of problem-solving that teenagers will face when they leave school.
In the ‘real world’, if you were posed that chicken pie problem (and by the way, I’m prepared to bet you will NEVER encounter it in the real world) then you would do what all good problem-solvers do: ask the person who posed the problem what they meant by it, and why they need an answer.
Alas, exam questions don’t offer any opportunity to check understanding. Nor do they allow another vital problem solving technique, which is to sleep on it. Or at least to take a break, have a cup of tea, and come back to the problem with a fresh mind later. In that respect, at least, my own exam experience was more realistic, though instead of a cup of tea, it was twenty minutes reading a chapter from Harry Potter to my kids.
P.S. Schools and exam boards are understandably reluctant to publish recent GCSE exam questions, because they are used extensively for mock exams. I respect that policy. I have made an exception with the chicken pie question because I think it needs some wider exposure. This is a poor question that does a disservice to the next generation of statisticians.