ofsted

Playing the Game

e=mc2andallthat4 Comments

Kings made tombs more splendid than houses of the living and counted old names in the rolls of their descent dearer than the names of sons. Childless lords sat in aged halls musing on heraldry.

— J. R. R. Tolkein, The Two Towers

image

If there’s anything that makes me lose the will to live, it is being in the same room as an educational Player of Games. I’m sure everyone is reasonably familiar with the type: “I want every intervention from now until the end of term focused on improving the A*-C pass rate for left-handed Y11 students whose birthday month has an R in it.”

Yes, it might help, marginally, in some sense. On such massaging of the margins are modern educational careers and reputations built.

Personally, such considerations leave me cold. Such teachers, it seems to me, hold their statistics in higher esteem than their students. The percentage is adjudged to be the outcome, rather than merely an indicator of a number of successful outcomes.

Sometimes, when I try and express this, people look at me as if I had twelve heads. It is a nuanced and subtle difference of emphasis, admittedly, but I think it’s a valid one. As an analogy, imagine a doctor who focuses on (say) a patient’s temperature to the exclusion of all else: “Doctor, I think I’ve broken my leg.”

“H’mm, let’s have a look. Actually, your temperature is a wee bit high. Here, let me apply this cold compress to your forehead.”

“But what about my leg?”

“Well, your body temperature is back to normal now. That means that we now have the officially mandated number of ‘healthy’ patients as per Ofdoc guidelines.”

“But what about my bloody BROKEN LEG?”

“My work here is done. Next patient please!”

The other note of caution that needs to be sounded more loudly in the education world is awareness of what is known as the Halo Effect.

I learned about this in Duncan Watts’ excellent book Everything Is Obvious (When You Know The Right Answer) in which he summaries the work of Phil Rosenzweig:

Firms that are successful are consistently rated as having visionary strategies. strong leadership, and sound execution, while firms that are performing badly are described as suffering from misguided strategy, poor leadership or shoddy execution. But, as Rosenzweig shows, firms that exhibit large swings in performance over time attract equally divergent ratings, even when they have pursued exactly the same strategy, executed in the same way, under the same leadership all along. Remember that Cisco Systems went from being the poster child of the Internet era to a cautionary tale in a matter of a few years . . . Rosenzweig’s conclusion is that in all these cases, the way firms are rated has more to do with whether they are perceived as succeeding than what they are actually doing.

— Watts, p.197 [emphasis added]

In one early experiment, several teams were asked to analyse the finances of a fictitious firm. Each team was rated on their performance and then asked to evaluate their team in terms of teamwork, communication and motivation. The high scoring teams assessed themselves very highly on these metrics compared with the low scoring teams, as you might expect. However, the kick was that performance scores had been allocated at random — there was no real difference between the teams’ performance at all. The conclusion is that the appearance of superior outcomes produced an illusion of superior functionality.

Watts argues persuasively that we tend to massively underestimate the role of plain, dumb luck in achieving success. He cites the case of Bill Miller, the legendary mutual fund manager who did something no other mutual fund manager has ever achieved: he beat the S&P 500 for fifteen straight years. Watts notes that this seems a classic case of talent trumping luck. However:

. . . right after his record streak ended, Miller’s performance was bad enough to reverse a large chunk of his previous gains, dragging his ten-year average below that of the S&P. So was he a brilliant investor who simply had some bad luck, or was he instead the opposite: a relatively ordinary investor whose ultimately flawed strategy just happened to work for a long time? The problem is that judging from his investing record alone, it’s probably not possible to say. [p.201]

I trust that I do not have to draw too many lines to highlight the relevance of these points to the education world. Outcomes, in the sense of exam grades, are currently the be-all and the end-all of education. But the Halo Effect makes it clear that a simplistic reading of successful outcomes can be highly misleading.

Negating the Halo Effect is difficult, because if one cannot rely on the outcome to evaluate a process then it is no longer clear what to use. The problem, in fact, is not that there is anything wrong with evaluating processes in terms of outcomes — just that it is unreliable to evaluate them in terms of any single outcome. [p.198]

Ofsted. managers and politicians please take note: our search for a signal continues.

Approval of what is approved of
Is as false as a well-kept vow.

— Sir John Betjeman, The Arrest of Oscar Wilde At The Cadogan Hotel

Meet The New Ofsted, Same As The Old Ofsted

e=mc2andallthatLeave a comment

During CPD training in school, the team was handed a bulging A4 booklet. So bulging, in fact, that the staples looked to be experiencing the same kind of tectonic stresses as the waistband of my work trousers during one of my ‘heavy’ phases.

I am sure that all teachers have a been handed such a booklet at some point. It was a collection of Powerpoint slides — printed on that setting that produces a set of lines next to a shrunken facsimile of each slide. The lines are generously provided for the lucky attendee of external CPD to write “Notes”. (Somewhere in that corner of a higher dimension known as Tree Heaven, one tree turns to another tree and says “Bastards! They cut us down for that?”)

Handing us a copy of the Powerpoint, of course, serves a double purpose: (a) the external-CPDer can tick the “info. shared with dept.” box on the yellow CPD Impact Assessment Form; and (b) it keeps the team occupied for twenty minutes as we digest the slides. The document itself was no worse than many I’ve seen, but, sadly, no better either: Ofsted…Ten things to remember…Ofsted..five strategies to…more Ofsted…six sodding hats…yet more Ofsted…bloody Bloom’s bloody taxonomy…[epithets mine].

But I digress. The potted biography of the trainer was included: she was headteacher there and there and is an experienced Ofsted inspector. Now I’m sure she is a nice lady who means well and gets on with her colleagues and family and doesn’t kick her cat and takes good care of the hamster, but I couldn’t shake the feeling that this sort of thing is beyond a cottage industry now. Now it’s an industry — the school improvement industry.

Down with this sort of thing

It’s like we went to bed in the green, bucolic splendour of the 18th Century and woke up amidst the hideous, belching smokestacks of the Industrial Revolution.

And, for the life of me, I could not shake the feeling that some paragraphs written by George Orwell in the 1940s were particularily relevant:

The corruption that happens in England is seldom of that [conscious] kind. Nearly always it is more in the nature of self-deception, of the right hand not knowing what the left hand doeth. And being unconscious, it is limited . . . I do not suppose there is one paper in England that can be straightforwardly bribed with hard cash. In the France of the Third Republic all but a very few of the newspapers could notoriously be bought over the counter like so many pounds of cheese. Public life in England has never been openly scandalous. It has not reached the pitch of disintegration at which humbug can be dropped.

— George Orwell, England, Your England

I am sure that there is not a single Ofsted inspector in the country who can be bought across the counter for cold, hard cash like so many pounds of cheese. I even accept that a recent Ofsted rule change means that serving inspectors cannot run “what Ofsted want”-style courses anymore. (And about time too.)

But is it enough? Will there simply be a time-delayed revolving door between a stint as an inspector and joining the school improvement gravy train? I suspect that the niceties will continue to be observed, and that the fine old traditional British value of humbug will stop the development of situations that are openly scandalous.

As a colleague observed cynically: “The people writing this kind of thing are the exactly same kind of people who will be judging us, and can make or break our careers. Don’t do as they do, do as they say.”

I am and I will continue to do so. But, openly scandalous or not, I still think it stinks.

Wilshaw’s “Block of Wood” Moment?

e=mc2andallthat1 Comment

Old Andrew draws our attention to an apparent turnaround in the Ofsted framework which make them less judgemental of traditional, didactic teaching techniques.

Inspectors must not give the impression that Ofsted favours a particular teaching style. Moreover, they must not inspect or report in a way that is not stipulated in the framework, handbook or guidance. For example, they should not criticise teacher talk for being overlong or bemoan a lack of opportunity for different activities in lessons unless there is unequivocal evidence that this is slowing learning over time.

— Ofsted [1]

I have mixed feelings about Sir Michael Wilshaw, the instigator of this change. On the one hand, I have been dismayed with comments like “if anyone says to you that ‘staff morale is at an all-time low’ you will know you are doing something right.” [2]. On the other, in some speeches it seemed that he was attempting to address the dead hand of trendy “group-work-good, teacher-talk-bad” Ofsted orthodoxy.

In a previous post, I likened Wilshaw to the captain of a supertanker, and asked whether he would be able to rein in the “talk-less-teaching” rottweilers on his staff. Large organisations can have a momentum as stubborn as supertanker and plough onwards in the same direction for mile after mile, whatever the frantic signals from the wheelhouse say.

In this latest iteration of the ever-changing whirligig that is the Ofsted inspection framework, Wilshaw appears to have nailed his colours to the mast. Rather than “T” for “Trendy” it seems that he is flying the “P” for “Pragmatic” flag — anything goes, as long as it works.

And it is a change for the better, as long as inspection teams adhere to the guidance. (How we can judge exactly what works is another can of worms that I don’t propose to dig into here.)

Could this be the defining moment for Wilshaw? Possibly, it could be his “block of wood” moment.

Knowing also that the severities of the past had earned him a certain amount of hatred, to purge the minds of the people and to win them over completely he determined to show that if cruelties had been inflicted they were not his doing but prompted by the harsh nature of his minister. …. [T]hen, one morning, [the minister’s] body was found cut in two pieces on the piazza at Cesena, with a block of wood and a bloody knife beside it. The brutality of this spectacle kept the people of the Romagna for a time appeased and stupefied.

— Niccolo Machiavelli, “New principalities acquired with the help of fortune”, The Prince

 

The reference is to a story about Cesare Borgia and how he calmed a turbulent province by appointing a “cruel and unscrupulous man” to rule as his minister. When the man’s severe methods had secured a measure of peace and calm to the territory, Cesare Borgia dispensed with his services in the very final and bloody way outlined above.

Just to be clear, I am likening Wilshaw to Borgia and saying that he has, essentially, dispensed with the services of the previous “cruel and unscrupulous” style of Ofsted framework in a very public way. OK, so it’s by way of pressing the delete key on a keyboard rather than a bloody knife, but I hope you get my drift.

Every headteacher I have ever worked with has been a consumate politician (which is not automatically a bad thing, by the way), and I cannot help but wonder if this is only a part of a “Great Game” being played out. (Unlikely, I know, but it’s fun to speculate: the more probable, pedestrian truth is that, as a character in All The President’s Men observed, “they’re just not that bright.”)

And so is the teaching profession “appeased and stupefied” by Wilshaw’s action? A little bit, perhaps. For my part, I will wait and see what effect this has on the next round of inspection reports and (perhaps more importantly) internal school observation criteria before I celebrate, but I am, I must confess, faintly encouraged.

And on that cheerful note: Happy New Year!

[1] http://www.ofsted.gov.uk/filedownloading/?file=documents/inspection–forms-and-guides/s/Subsidiary%20guidance.pdf&refer=1

[2] http://www.tes.co.uk/article.aspx?storycode=6145814

Coe, Wilshaw and Ofsted

e=mc2andallthat3 Comments

[T]he community suffers nothing very terrible if its cobblers are bad and become degenerate and pretentious; but if the Guardians of the laws and state, who alone have the opportunity to bring it good government and prosperity, become a mere sham, then clearly it is completely ruined.

— Plato. The Republic 421a (Penguin Classics) (Kindle Locations 2815-2817). Penguin Books Ltd. Kindle Edition.

I am not sure if I agree with Plato about cobblers and the community. As Benjamin Franklin once pointed out: “For want of a shoe…the kingdom was lost.”

However, I think his statement about the Guardians of the state stands. Equally so with the state-appointed guardians of education: Ofsted.

I think Professor Coe (http://www.bbc.co.uk/news/education-24079951) has done the education world a huge service by pointing out that the Ofsted king has no clothes. However, perhaps in the manner of scrupulous academics everywhere, Professor Coe might prefer a more nuanced “the king probably doesn’t have any clothes”.

Coe pointed out that there is no — repeat, no — valid research supporting the “Ofsted model” of classroom observation being either: (a) a reliable tool for assessing teaching quality or effectiveness when cross-referenced with other measures such as student learning gains; or (b) the observation-feedback process leading to an improvement in teaching quality. (See  http://www.youtube.com/watch?v=_-UyGwYHhGY for the section of his talk on classroom observations and http://t.co/AqY7Xqzknw for a link to his slides.)

I don’t know about anyone else but I am staggered by this. As a working teacher who is just about maintaining a precarious foothold on the treacherous scree of middle management, I always thought my seniors and betters had reams of evidence supporting the stuff they were asking us to do. And if they didn’t, well probably their seniors and betters did.

To hear a respected academic say that classroom observation might be “the next Brain Gym” was shocking.

And the Ofsted response? “Tosh and nonsense,” said Sir Michael Wilshaw.  “I don’t know of any headteacher who doesn’t believe that classroom observation isn’t anything other than a help. The fact that we are an inspectorate and we do make judgements has made a huge amount of difference.” According to the TES (13/9/13 p.8):

He said that new figures released this week, showing a 9 percentage point rise in the proportion of schools judged to be good or outstanding, proved that the watchdog’s tougher inspection regime had “galvanised the system”

This is, to my mind, a textbook example of the logical fallacy known as petito principii or “circular reasoning”. The form of this particular logical fallacy is as follows:

Logical Form:

     Claim X assumes X is true.

     Claim X is therefore, true.

Bennett, Bo (2012-02-21). Logically Fallacious: The Ultimate Collection of Over 300 Logical Fallacies (p. 82). eBookIt.com. Kindle Edition.

Let’s see what Wilshaw said again.

  • The inspectorate’s judgements make a huge amount of [presumably positive] difference.
  • Ofsted judgements show that more schools (9 percentage points!) are good or outstanding.

…therefore Ofsted judgements make a huge amount of positive difference.

Now please note that this does not necessarily mean that the conclusion (that Ofsted helps schools improve) is false, but merely that the argument put forward by the Chief Inspector to support that conclusion is fallacious. And it hopefully goes without saying that a fallacious argument is by definition invalid and must be dropped immediately.

The character Chief Brody in the film Jaws once remarked that they needed “a bigger boat”. The Chief Inspector needs a better argument. And in view of the large amount of taxpayers’ money going to support Ofsted, that new argument should be supplied sooner rather than later. As Professor Coe remarked (somewhat plaintively) in his excellent talk: “Just one would be nice.”

H’mm. More rigour, anyone?

The Woman Who Is Kicking the Hornets’ Nest

e=mc2andallthat3 Comments

So, I’m reading  Seven Myths about Education.  Just like most of the rest of the teaching blogosphere, I suspect. And just like most of the rest of the teaching blogosphere, I have an opinion about it. Several, as a matter of fact. And since I am now about halfway through, I thought I’d share my thrupence’ worth.

To begin with, is Ms Christodoulou more like the boy who cried that the king had no clothes or the boy who cried wolf?

For my money, she is more the former than the latter. I think the estimable Ms Christodolou is calling time on some pretty dodgy ideas.

Some ideas are as ubiquitous and seemingly essential as air, but as Joseph Joubert correctly opined: “A thought is a thing as real as a cannonball”.  And in some circumstances, the wrong idea can be more dangerous than a large round metal ball travelling at close to the speed of sound.

Now teaching-wise, I have to confess that I have been around the block a few times. I am the definitive “old fart in the staffroom”. Like many old farts, I could bring myself to believe that oftentimes it is not what Ofsted actually said that was the main problem, but what all-too-many people thought that Ofsted said: some half-remembered, half-digested soundbite from some godforsaken half-decade-old CPD.

Christodoulou marshals some convincing evidence that often it is the actual demands of Ofsted that create the problem. It seems that Ofsted genuinely do not like didactic teaching, and we’re not just imagining it. Christodoulou presents some damning examples of the current vogue of trashing “teacher talk” from inspection reports. Whether Wilshaw will be able to rein in the “talk-less-teaching” rottweilers on his staff is open to debate. Large organisations can have a momentum as stubborn as supertanker and carry on going in the same direction for mile after mile, whatever the frantic signals from the wheelhouse say.

One of the passages that resonated most strongly with me was this:

For example, in a project that involved pupils writing any type of extended writing … I would provide them with a helpsheet summarising what they should put in each paragraph. […] Rather than breaking down the individual components required to write good reports and teaching those, I was asking students to write a report and then giving them a few cheats or hints about how to do it. It is rather like teaching pupils a few cheats or hints that would help them play a certain song on the piano, while neglecting to teach them the scales and musical notation.
— Location 1727, Kindle edition

Been there, done that, smugly uploaded the worksheet on to the TES Resources website…

She quotes psychologist Dan Willingham: “the most general and useful idea that cognitive psychology can offer teachers [is to] review each lesson plan in terms of what the student is likely to think about.”

Christodoulou argues that teaching (say) Romeo and Juliet by getting the students to make fingerpuppets of the main characters is counterproductive because the students spend more time thinking about making fingerpuppets rather than Romeo and Juliet. “That is not to say that … puppetmaking [is] unimportant. The problem is that this lesson . . . was supposed to be about Romeo and Juliet. If the aim of the lesson was … how to make a puppet, it would have been a good lesson. Not only do these types of lesson fail in their ultimate aims, but because they are so time-consuming, they also have a very significant opportunity cost.”

I agree with Christodoulou that direct instruction is often the most effective form of teaching. Now don’t get me wrong, I am not proposing that teachers spend the whole of the lesson talking at their charges. What I am saying is that students’ thinking should be channelled to engage as directly with the concepts being taught as possible. And at the heart of good teaching is clear, succinct, unhurried teacher talk.

The fingerpuppet stuff I have done, but only to pass observations. Sadly, honesty is not the best policy these days.

A while back, Arnold Schwarzenegger was The Terminator: robot on the inside, human on the outside.

Call me the The Didactor: steely-eyed, garrulous, “I’ve-got-a-banda-and-I’m-not-afraid-to-use-it” old-school (hah!) schoolteacher on the inside; cuddly, Ofsted-friendly, near-mute “lesson-facillitator” on the outside (readers of a certain generation are invited to think of a cross between Fingerbobs and Marcel Marceau).

Sigh, I wish. I got a 3 (“Requires improvement”) in my last lesson observation.

The secret of success is sincerity. Once you can fake that you’ve got it made.

— Jean Giraudoux

More sincere faking is required on my part, I feel.