This post is the fourth in a series of outcomes from my MA in Education work. In the first post, I looked at why might schools use research? In the second post, I looked at what makes up a teaching and learning culture, and in Part 3, I looked at how people generally view education research in comparison to other factors that shape it. In this post, I look at how that attitude changes depending on the extrinsic situation you’re in!
It’s a brief discussion – and I welcome all thoughts & ideas from it! It’s highly tentative, and comes with disclaimers and context as described.
Extrinsic Motivator: Examination Results
Having identified “better results” as the most important motivator in the use of educational results from my sample, further analysis is now conducted to explore the dimensions of that experience in creating T&L cultures. All respondents are split according to their self-described results, with the table showing the distribution of these groups. Any respondents who did not answer this question are removed from further analysis in this section.
|Description of Results||Respondents||% of Respondents|
|Significantly better than local schools||72||24.9|
|A bit better than local schools||86||29.8|
|About the same as local schools||75||25.9|
|A bit worse than local schools||39||13.5|
|Significantly worse than local schools||17||5.9|
Table 1: Distribution of responses by self-described categorisation of examination results
Responses are standardised by category, and presented as percentage of the responses in an individual category: while this potentially skews analysis to a greater percentage for the lower categories, it enables easier comparisons to be made between schools with different results.
With the collective response showing the importance of results, the original hypothesis was for the general trends to reflect grading differences; in other words, schools with better results would have better teaching and learning cultures and feel more positively about them, and perhaps by association, their use of educational research.
Figure 19: Chart to show the importance of examination results, sorted by category of exam results
As the primary motivator of collective responses using educational research, the categorical response to the same question is interesting. All responses in the “significantly worse” category, and all but 13% of the “a bit worse” group rate the use of educational research as very or fairly important in driving their school teaching & learning culture. However, there is no significant disparity between the attitudes: almost all categories show consistently higher than 85% positive relationship with the use of research to generate results.
Figure 20: Chart to show the belief in educational research as a driver of exam results, sorted by category of exam results
However, when we look at the impact of that process – rather than the motivation – there is more of a shift. Figure 20 shows that schools who have better/significantly better results appear to have fewer positive associations with education research: perhaps they can identify other factors to which they can attribute their exams success, whereas schools who identify as “same” or “a bit worse” are dominated by the positive responses. It is interesting that the schools who describe themselves as “significantly worse” do not seem to have full confidence in the impact: 25% of respondents are neutral or disagree that educational research is effective.
Figure 21: Chart to show the importance of using educational research to satisfy inspectors, sorted by exam categories
If we look at the Bernhard et al. (2020) hypothesis which suggests schools use evidence as an anchor (or shield) against inspection judgements, it is interesting to note that there is no single response that says “not at all” in any category. Figure 21 shows the inspection process clearly has some weight, even if exam results are uppermost in responses, and judgements of importance. The highest proportion of responses “for Ofsted” appear to be done in the areas doing a little worse than local schools (75% positive), just ahead of those doing significantly better (68% positive). These two groups are notably above the other three categories; and indeed, the lowest ranked category of exams results appears to be least likely to use educational research to satisfy external inspectors. A similar result is shown in the relationship between educational research and parental/Governor satisfaction, with the analysis not shown here for brevity. Schools where exam results are significantly worse appear to have limited interest in the utility of research to satisfy external stakeholders; with a majority neutral opinion, and no regard for it as very important. As school exam results improve, the trend is for the views of stakeholders to be more positively regarded in the use of educational research.
Figure 22: Chart to show the relationship between descriptions of teaching and learning culture, sorted by exam categories
It is important to consider whether the extrinsic and intrinsic motivations for utilising educational research overlap. In this section, some of these intrinsic elements are unpicked further.
First, Figure 22 explores the relationship between the description of teaching & learning culture and the exam results. In general, it seems that the culture is described more positively where the exam results are better – potentially through either causal mechanisms of validating the efforts made by staff, or by creating a positive culture leading to good outcomes. There is no evidence to suggest either of these is more likely than the other!
The least positive relationship appears to be in places where the results are described as “a bit worse than local schools” which may well reflect changes and local challenges, or the imposition of a “turn around” culture and the potential tensions therein.
Figure 23: Chart to show attitudes towards educational research and teacher development, sorted by exam category
Figure 23 shows that the reflection of utility of research for personal development similarly shows an unexpected relationship. The highest confidence in personal development through the use of educational research comes in schools whose results are comparable to local counterparts (93% positive), and then there is a decline away from that down to 79% positive in those significantly better, and 77% positive in those significantly worse.
Figure 24: Chart to show attitudes towards educational research and workload, sorted by exam category
Similarly, when comparing an extrinsic factor to an intrinsic one, we see differences between schools, though this time at a different level. Figure 24 suggests that workload appears to be consistent between the top three categories of exam results, and there is little appreciable difference in the responses. However, the schools with worse exam results seem to split considerably, with a larger range of responses. For the “bit worse” category, the workload utility of research is the lowest of all sectors: only 54% positive rated, and the highest “not very important” at 23%. By contrast, the schools describing as “significantly worse” than local schools show the highest positivity for workload impacts: up to 74%, and the smallest neutral and not important ratings of all categories.
It seems that workload and processes are variable, and the pressures of schools in “turn around” status can often be significantly varied. It is beyond the scope and method of this enquiry to explore the experience of these examples; the originally intended method of triangulation interviews might have given further insight to the nature of work in these schools, but it offers a departure from this enquiry.
Figure 25: Chart to show the relationship between exam results and inspection results
Finally, Figure 25 shows the challenge of attempting to disentangle the motivations of extrinsic factors as identified in the limitations section.
There is a clear relationship demonstrated between exam results and inspection outcomes: 89% of the schools with significantly better results are graded as outstanding or good; while only 29% of the schools with significantly worse grades are graded so. Overwhelmingly, those schools are rated as RI (30%) or inadequate (41%). The challenge of correlation vs causality, and the inability of the quantitative data to unpick rationales is demonstrated quite clearly here: we can’t interrogate each enquiry to understand the motivations, and have a triangulation conversation to explore how and why each of these relationships are linked to their context.
This leaves the potential conclusion that the extrinsic motivation of examination results is important, and that schools in the edges of the median position seem to feel this pressure most significantly. They have the most positive attitudes towards the ability of education research, the impact on outcomes and seem to accept the workload consequences!
Accepting this co-variability and the potential limitations of the extrinsic factors overlapping, all respondents were split and analysed by their self-described inspection grades, to explore whether there were more obvious trends apparent.
|Rating||Number||% of Schools|
|Outstanding (OFSTED)/Excellent (ISI)||81||28.8%|
|Good (OFSTED, ISI)||150||53.3%|
|Requires Improvement (OFSTED)/Sound (ISI)||35||12.5%|
|Inadequate (OFSTED)/Unsatisfactory (ISI)||15||5.3%|
|Prefer not to say||23|
Table 2: Distribution of Responses by Inspection Category
For the purpose of simplicity, the OFSTED terminology will be used hereafter to describe both categories of school inspection.
The majority (53.3%) of schools self-described as “good”, with about a third as “outstanding” and the rest split between RI and Inadequate status. 23 respondents preferred not to disclose the information, and were discounted from further analysis as a result.
As before, these varying categories are scaled to show standardised comparisons, noting and accepting the potential distortion of oversampling the lower categories.
Figure 26: Chart to show the attitude towards results in creating teaching and learning culture, sorted by inspection category
Building on the previous relationship between results and inspection category, Figure 26 shows that inadequate and RI category schools rate the use of education research as almost entirely to drive better results. While there is minor difference in the positive ratio for good/outstanding schools, there is less criticality: more define it as “fairly important” rather than very important as a reason to engage with education research.
Further analysis (graph not shown) suggests that there is a relatively consistent belief in the efficacy of the utility of educational research, where 65-75% of respondents in all categories describe a positive belief that research drives results. Only a small fraction (less than 5%) believe that there are no impacts on student results. The consistency of purpose, rather than intrinsic belief in the implementation of educational research, tends to suggest that the extrinsic motivation for using it is more significant than the intrinsic judgement of whether it yields the right outcome.
Figure 27: Chart to show the confidence in the use of educational research, sorted by inspection category
This trend appears to be continued through the confidence in research analysis in Figure 27.
While not as stark, there does appear to be a small trend in the confidence in the use of research as employed by the school as an inverse relationship with inspection outcomes. While only 36% of outstanding schools are confident, nearly 54% of the inadequate schools describe their confidence in the school’s use of research positively.
Further evidence (not shown) shows a consistent perception across all categories in the efficacy of research in personal development; suggesting that the use, and the teachers’ role in the process is less important than some of the contextual elements (results, judgement status).
This may well link to the appointments and structural themes identified earlier, including the designation of Research or Evidence Leads, and might be reflected in further study and analysis.
Figure 28: Chart to show the extent of educational research utility, sorted by inspection category
This perhaps connects to the perceived utility of educational research (Figure 28), with inadequate schools again showing a much greater positive response (73%) compared to RI (57%) and outstanding (47%) or good (41%).
The response rate for negatives (disagree/strongly disagree) remains reasonably consistent across all four categories; what changes appears to primarily be the extent to which people have “neutral” opinions on this question: from 24% neutral in a good category, through to none in the inadequate.
This may be an interesting result to explore further; but could also be a limit of the comparatively small sample size of the inadequate category.
Figure 29: Chart to show the relationship between teaching and learning culture and the satisfaction of external inspectors, sorted by inspection category
The previous link between exam results and external stakeholders is reinforced through further analysis of Figure 29, and a similar pattern found. Again, not a single respondent believed that there was no impact of external inspectors on the use of research.
There is little difference between the outstanding/good schools in their rationale relative to external inspectors, and they are fairly close to the inadequate schools, too (55% important in the outstanding/good vs 40% important in the inadequate category).
However, the schools ‘on the margin’ of requires improvement appear to be most driven by the OFSTED rationale: with 77% of respondents believing that the extrinsic inspection rational was fairly or very important. A similar trend to the exam results is shown in the satisfaction of parents and governors (data not shown here). All but the ‘inadequate’ category showed relatively consistent attitudes to the use of research to satisfy parents. Inadequate schools regard the use of research to satisfy parents as relatively unimportant: 42% as “not very important” and 50% as neutral.
In terms of extrinsic motivators, then, it seems that Ofsted and results are inextricably linked, and are the key external drivers of the use of educational research in schools.
Figure 30: Chart to show the extent to which participants believe in the impact of education research on workload, sorted by inspection category.
Of the potential variables, the most significantly demonstrated intrinsic motivation for schools is the efficacy of education research in the ability to improve workload for teachers.
Although some observable disparities exist (Figure 30), there is some consensus across the three top categories. Inadequate schools stand out again, as the exception to the wider trend: with a very divided response (no neutral opinions), and a 70:30 split in terms of in favour vs disagreement.
The interpretations of the data sorted by external judgement and exam results would appear to support the conclusion that educational research is used to satisfy extrinsic motivation, and agree with Bernhard et al. (2020)’s hypothesis of research being used as a “shield” for judgements by schools in the two lower categories. We see similar levels of confidence and understanding about the efficacy of research and how it translates in to impact, but a greater utilisation of educational research as a mechanism to build teaching and learning cultures in schools.
It would be interesting to explore the extent to which these schools also have a strong leadership team, or an appointed “Teaching and Learning Lead”, because the relatively high confidence in how educational research is being interpreted and applied to setting does not relate to the prior training and experiential questions. This might generate a proxy of a research-informed culture, in which participants are presented with research through the prism of someone else’s leadership or experience. They are confident in the selection of material they are being presented with, and have all read the same papers/attended the same sessions, but may perhaps be unaware of the wider limitations, complexities and debates in the field – which
However, it could also be argued that in situations of high extrinsic need (poor exam results, low inspection status), the need to prioritise and focus on a specific outcome or target addressing one (or both) of these needs might well over-ride the wider cultural development of schools. It is possible that schools with a single focus use the research as a lever to move that focus, and this reduces other extraneous and plural perspectives in the day to day teaching. This is an equally plausible explanation of the analysis; which, without further triangulation research is uncertain.
The extrinsic motivations show real drivers of the use of educational research, and of the creation of teaching and learning cultures. However, to address the final research question, it is also important to consider intrinsic motivational factors.