Ofsted methods may not be valid, says senior academic

Schools inspectors in England are basing their verdicts on evaluation methods which may not be reliable, a senior academic has claimed. This is from the BBC

Prof Robert Coe, whose work has been cited by ministers, said there was no proof that Ofsted’s lesson observations led to valid judgments.

Prof Coe said inspectors relied on lesson observation despite no evidence that it led to better learning.

Ofsted chief Sir Michael Wilshaw called the claims “tosh and nonsense”.

“I don’t know of any head teacher who doesn’t believe that classroom observation isn’t anything other than a help,” said Sir Michael in an interview with the Times Educational Supplement.

He drew on figures released on Monday by Ofsted showing a nine percentage point rise in the proportion of schools judged good or outstanding and said this proved the watchdog’s tougher inspection regime had “galvanised the system”.

“The fact that we are an inspectorate and we do make judgments has made a huge amount of difference.”

Prof Coe, director of Durham University’s Centre for Evaluation and Monitoring, told a major educational research conference that Ofsted needed to demonstrate its lesson observations were valid and backed by research evidence.

He added that millions were being spent on the watchdog in the hope of raising educational standards but questioned whether inspections benefited the system.

He said some studies in fact suggested that schools take a long time to recover from inspections and they don’t do any good.

He told BBC News: “If you sit in a classroom, everyone thinks they can judge how good the lesson is – but can you really?

“Quite a lot of research says that you can’t.”

He added that schools increasingly use classroom evaluation to monitor lesson quality internally.

“Teachers do it every week – but is it reliable?

“Teachers are trying to do what Ofsted wants – but does it really make things better?

“If Ofsted says a lesson is no good – that’s a big deal.”

In his talk, Prof Coe suggested that ratings given to lessons by observers could be “influenced by spurious confounds”.

These included the charisma and confidence of the teacher, the subject matter being taught, students’ behaviour in the classroom and the time of day.

He questioned whether the observation ratings could be consistent given so many variables.

He also listed a series of “poor proxies for learning”, arguing that outward signs such as busy and motivated students and a calm and ordered classroom do not necessarily always mean that students are learning effectively and could reproduce correct answers independently.

More at:  Ofsted methods may not be valid, says senior academic

What do you think of these criticisms of Ofsted inspections, especially in relation to classroom observations? Are they valid and consistent or too easily influenced by ‘spurious confounds’ as Professor Coe suggests? What kind of evaluation do you think would be more effective? Let us know…

New free school 'causes class sizes rise' at nearby state school
Parents should not be distracted from children by mobiles, say headteachers
Categories: Policy and Research.

Comments

  1. jdlichfield

    SchoolsImprove he has a point and I was not surprised by Wishaw’s “closed mind” response. Ofsted inspections have their limits and “issues”

  2. Siobhan_Heffron

    SchoolsImprove regular ‘pop in’ style obs allow you to have a clear overview of what really happens. Not one show off lesson!

  3. acet2001

    wonderfrancis NUTonline SchoolsImprove “Tosh and Nonsense” and “Yadda Yadda Yadda” just about sum up Gove and Wilshaw. What a pair!

  4. andylutwyche

    SchoolsImprove Finally Ofsted’s flaws are being highlighted. Wilshaw well aware of them and so offers a Govian response. Ofsted end nigh?

  5. educationbear

    Siobhan_Heffron SchoolsImprove Agree! Have had success with this style of obs matched with more formal obs at other times.

  6. acet2001

    jdlichfield SchoolsImprove “The limits and issues of OFSTED inspections” would resemble War and Peace in size but without the humour!

  7. cparkie

    RufusWilliam SchoolsImprove wonderfrancis I thought OFSTED were looking for confirmation of the school’s declared teaching provision.

  8. drmattoleary

    andylutwyche SchoolsImprove Have been arguing the case for some time. Next step is for colleges/schools to challenge their judgments

  9. andylutwyche

    drmattoleary SchoolsImprove Got to have HTs and governors with a lot of nerve to do that as they risk a poor judgement next time then out

  10. andylutwyche

    drmattoleary SchoolsImprove It does need to happen though, and if Ofsted blank challenges then refusal of access to inspect next step?

  11. andylutwyche

    acet2001 SchoolsImprove It does smack of bullyboy, and apparently not out of character. The response is straight out of Gove’s phrase book

  12. drmattoleary

    andylutwyche SchoolsImprove Spot on. Requires courage on the part of SLT/SMT to support & defend what their staff do & know what’s best

  13. livelikeatticus

    SchoolsImprove observations are just one part of a much bigger jigsaw. Yes they help see whole picture, but they’re not the only piece.

  14. murphiegirl

    andylutwyche SchoolsImprove drmattoleary Good HTs can demonstrate clear knowledge of their schls & staff without need of divisive grades

  15. murphiegirl

    andylutwyche SchoolsImprove drmattoleary Good HTs can demonstrate clear knowledge of their schls & staff without need of divisive grades

  16. andylutwyche

    drmattoleary SchoolsImprove Nothing will change until it happens – judgements on a whim can hugely effect funding etc. Legal challenge?

  17. andylutwyche

    murphiegirl SchoolsImprove drmattoleary Too many accounts of Ofsted findings not matching actuality for there not to be alarm bells

  18. drmattoleary

    andylutwyche SchoolsImprove Sorry state of affairs to resort to legal challenge. Ideally should be based on professional dialogue

  19. drmattoleary

    andylutwyche SchoolsImprove Though recognise, of course, that current inspection regime doesn’t do ‘dialogue’. Ofsted judge & jury

  20. andylutwyche

    drmattoleary SchoolsImprove Agreed, but Ofsted/DfE don’t have a great recent record of listening to counter-arguments. Worst case scenario

  21. drmattoleary

    andylutwyche SchoolsImprove If improvement really is the key driver then the voices of all involved need to be heard rather than just one

  22. murphiegirl

    andylutwyche SchoolsImprove drmattoleary Actually heard an Inspector admit in CPD, when faced with choice between G & O judgment …

  23. murphiegirl

    andylutwyche SchoolsImprove drmattoleary …’Depends what mood I’m in and if I like the teacher.’ Stunned silence from the room.

  24. andylutwyche

    murphiegirl SchoolsImprove drmattoleary Unbelievable! It does give some weight to the accusation of judgement made in car on way to schl

  25. drmattoleary

    murphiegirl andylutwyche SchoolsImprove Great quote Rachael! Have heard similar things from observers who talk about ‘gut instinct’ being

  26. drmattoleary

    murphiegirl andylutwyche SchoolsImprove key to identifying ‘excellent’ teachers. May be some truth in that but if that’s the case then

  27. andylutwyche

    drmattoleary murphiegirl SchoolsImprove Ah, the old “gut instinct” chestnut – I know it so well and it’s so scientific

  28. drmattoleary

    murphiegirl andylutwyche SchoolsImprove then we need to call time on this pretence of making so-called ‘objective’ judgements

  29. andylutwyche

    drmattoleary murphiegirl SchoolsImprove Problem is that most teachers would love to get rid of Ofsted but climate of fear too powerful

  30. drmattoleary

    andylutwyche murphiegirl SchoolsImprove Agree. It’s about transforming cultures of teacher assessment & development ground up & top down

  31. murphiegirl

    drmattoleary andylutwyche SchoolsImprove Absolutely – that’s why kevbartle ‘s Trojan Mice movement is so powerful – & so needed.

  32. murphiegirl

    drmattoleary andylutwyche SchoolsImprove Maybe this latest report will go some way to allow the StuffThatWorks (TM) room to gather pace

  33. murphiegirl

    drmattoleary andylutwyche SchoolsImprove StuffThatWorks = developmental, non-prejudicial/judgmental, empowering, evidence-based practice

  34. kevbartle

    murphiegirl drmattoleary andylutwyche SchoolsImprove Thanks Rachael. And why I need to get going with Trojan Mouse book project again.

  35. murphiegirl

    kevbartle drmattoleary andylutwyche SchoolsImprove Yes! Please count me in. Something happened with emails last round & I missed some

  36. kevbartle

    murphiegirl DM me your email address and I’ll add you to the mailing list and invite you to the Dropbox. Some good docs in there.

  37. murphiegirl

    drmattoleary kevbartle andylutwyche SchoolsImprove Get onto Kev’s blog via his Twitter account & you’ll be able to read all about it.

  38. sappg001

    SchoolsImprove problem I have is Ofsted follows a ticksheet and that is absolutely meaningless as far as actual learning is concerned

  39. LearnWLesley

    sappg001 SchoolsImprove I am always curious about those who become ofsted inspectors. Did they enjoy teaching? Were they good teachers?

  40. LearnWLesley

    stephhannaford sappg001 SchoolsImprove good to hear although is the antipathy I read towards them or machine within which they exist?

  41. stephhannaford

    LearnWLesley sappg001 SchoolsImprove quite honestly i guess that your reaction indicates not many are aware of this development.

  42. LearnWLesley

    stephhannaford sappg001 SchoolsImprove it was merely an observation having read others tweets since I started on twitter.

  43. ieshasmall

    murphiegirl totally believable. Once had a boss tell me it was “just that something. I can’t tell you how to move between the two.”

  44. Marshman

    The problem with inspection of public services is that improvement is always defined by the inspection body. Wilshaw says inspection works because ofsted has graded more schools as excellent! the Audit Commission was just the same – I worked there for six years. And is now largely abolished.

  45. murphiegirl

    ieshasmall Jobaker9 Even if you THOUGHT that, surely you’d cover it with a more believable reason. Otherwise you’d look like a dick. Oh…

  46. andylutwyche

    fionabw SchoolsImprove Absolutely right – Ofsted create hoops for schools to attempt to jump through, and that justify Ofsted’s existence

  47. vausekatie

    SchoolsImprove good teachers can get bad obs due to lack of guidance/clear direction from senior team. Clarity is needed on judgements

  48. andylutwyche

    Gwenelope murphiegirl SchoolsImprove drmattoleary Quite – and these people determine schools’ and teachers’ futures #shocking

  49. drmattoleary

    Gwenelope SchoolsImprove And this is not uncommon but Ofsted not interested in the longitudinal just the snapshot ‘performance’!

  50. andylutwyche

    Gwenelope murphiegirl SchoolsImprove drmattoleary Too much riding on the whim or mood of someone – no wonder people crumble

  51. andylutwyche

    Gwenelope murphiegirl SchoolsImprove drmattoleary If judgement based on the whim of observer then you have little influence over it

  52. drmattoleary

    Gwenelope SchoolsImprove Ofsted framework is illustrative of the systemic failure that continues to hinder genuine improvements in T & L

  53. murphiegirl

    Gwenelope andylutwyche SchoolsImprove drmattoleary I’ve also got to add that I personally know 3 inspectors (1 lead) who are the most

  54. murphiegirl

    Gwenelope andylutwyche SchoolsImprove drmattoleary scrupulously fair & honest people I know. They are all practising school leaders too

  55. drmattoleary

    murphiegirl Gwenelope andylutwyche SchoolsImprove Me too Rachael. The key issue is a SYSTEMIC flaw in inspection f/work not individuals

  56. murphiegirl

    SchoolsImprove In training inspectors isn’t it like teaching kids? Your (SMW’s) message & intentions might be spot on but how do you know

  57. murphiegirl

    SchoolsImprove they have learned & shared this properly? Can some insp too be ‘coached to the test’ but fail to apply learning afterwards?

  58. murphiegirl

    drmattoleary Gwenelope andylutwyche SchoolsImprove I think SMW’s intentions are honourable but, yes, system is flawed.

  59. P33 of the subsidiary guidance says the outcome of SMSC is confidence. A better measure of school effectiveness is to measure progress in achievement AND growth in confidence, as equal outcomes. Lesson this would re-focus the value of lesson observations.

Let us know what you think...