112
CURRICULUM, EFFECTIVENESS, EQUITY
Hungarian research project. It is important to emphasize disadvantage because in Hungary the connection between family background and student achievement is particularly strong (Andor – Liskó, 2000; Balázsi et al, 2014; Kertesi – Kézdi, 2010), and research also revealed that the school increases the differences rooted in the social background (Balázsi – Horváth, 2011).
ANIKÓ FEHÉRVÁRI: Educational Effectiveness and Disadvantage
FINDINGS
Figure 1 presents the composition of the student body the investigated schools.
The result of variance analysis clearly indicates a very significant difference from the nationwide average. The family background of the students in the investigated schools is five times worse, and the composition of the student body is nine times worse than the national average.
–6 –5 –4 –3 –2 –1 0
fam. bckgrd, avg
student body compos., avg nationwide
schools in integration programme
Figure 1 | Composition of the student body and students’ family background, 2014
Earlier studies found that schools where there is a high rate of (multiply) disadvantaged students essentially don’t have less qualified or experienced teachers but find it more difficult to fill the gaps in their staff and are lacking more teachers, consequently they have a poorer supply of specialised teachers (Varga, 2009;
Fehérvári, 2015). This is typical for the schools analysed: the number of unfilled teaching positions is far above the nationwide average and missing teachers often cause serious problems, and some of the subjects have to be covered by teachers whose major is in a different discipline. There is no difference in the educational value added of mathematics and reading comprehension in respect of teachers’
professional qualifications (the rate of university graduates was analysed); conversely, there is a significant difference between the value added of mathematics and reading comprehension when considering missing staff, supply of specialised teachers and teaching assistants. According to the result of variance analysis the school where all teaching positions are filled, subjects are 100% covered by the relevant specialised teachers, and there are teaching assistants who help teachers’ work represent higher value added. It is also conspicuous that it is the inappropriate supply of specialised teachers that has the strongest negative impact on value added. The difference between the two tested areas is also demonstrable: while mathematics appears to affect value added more positively, the effect of reading comprehension is mainly on the negative side.
114
CURRICULUM, EFFECTIVENESS, EQUITY
–20 0 20 40 60
all teachers w proper qualifications some teachers wo proper qualifications missing teachers no unfilled position Roma assistant
no Roma assistant educational added value –
reading
educational added value – mathematics
Figure 2 | Staff characteristics of schools, 2014
Schools’ external relations and embeddedness in the local community do not tend to be prominent issues in EER. Researchers generally look at parental involvement in this regard. However, when disadvantage is at the core of the study schools’
relationships are also important to consider primarily because of social welfare and employment both of which can have an important role in preventing social exclusion and extreme poverty. These sectors and their cooperation with education also play a part in the prevention of early school leaving and attrition. The questionnaire designed for school heads tried to explore several of these relationships and their intensity. We found that there was no correlation at all between value added and NGOs, family assistance service, non-school-based study halls, and professional and pedagogical assistance services. There are two exceptions, however. Schools that interact with the municipality and child welfare services on a daily basis are more effective than those that do not have such a regular interaction. The mechanism in the two areas measured is the same as described above: the effect is more positive on the mathematics related results and more negative on the reading comprehension results. In other words, schools that cooperate more closely with local organisations are more effective than those that don’t. A school appears as part of a complex system of services. It is important to note that the professional support system seems to have no detectable impact.5.
5 It is to be noted that the professional support system is currently being revamped.
ANIKÓ FEHÉRVÁRI: Educational Effectiveness and Disadvantage
–10 0 10 20 30 40
daily not daily daily not daily
child welfare servicemunicipality educational added value – reading
educational added value – mathematics
Figure 3 | Schools’ external relations, 2014
A spirit level of a school’s innovative capabilities is its successful application for grants in development schemes. Two types of development programmes were investigated: the SROP projects deployed over the past few years which were aimed at either educational content or infrastructure development; and other central initiatives that funded mainly enrichment type of extracurricular activities (Útravaló – MACIKA Bursary Programme, Eco School, Talent Point). Participation in such projects have a conspicuous positive effect on the school’s value added in both mathematics and reading comprehension. There is also evidence that programmes enriching school services have a greater impact on effectiveness.
–30 –20 –10 0 10 20 30
yes no yes no
SROP developmentOther central programme educational added value – reading
educational added value – mathematics
Figure 4 | Schools’ participation in development programmes, 2014
116
CURRICULUM, EFFECTIVENESS, EQUITY
As we had no data about the schools’ budget we asked the school heads which areas they thought to be underfunded6. Two of the eight areas mentioned have a significant correlation with value added. One has already been mentioned, and it has been found that enrichment programmes, i.e. the expansion of services have a positive effect on value added. Understandably, scarcity of funds in this area has a negative impact. The other important factor is teachers’ professionalism. Schools that have sufficient funds to finance continuing training of their teachers achieve a higher effectiveness score than schools where this area is underfunded. The two areas measured behave in a similar way in this respect but the difference in the mathematics value added is greater than on the reading comprehension side.
–40 –20 0 20 40
not enough funds enough funds not enough funds enough funds
teachers' continuing trainingexpansion of services
educational added value – reading
educational added value – mathematics
Figure 5 | Underfunded areas in the school as seen by school heads, 2014
We wanted to examine the combined effect of the factors detailed above, and to identify the factor that has the strongest influence on value added. Two linear regression models were prepared, one for the mathematics value added and one for the reading comprehension value added. The results were similar. Only one factor remained significant in the models: teachers’ continuing training. This means that in schools where funds are insufficient to ensure teachers’ in-service training there is a greater likelihood that the value of educational value added will be negative compared to schools that are capable of funding continuing training. Both models have medium explanatory power; the beta coefficient is slightly higher for the mathematics value added than in the reading comprehension side.
6 Eight areas were mentioned: basic functioning, building renovations, installations, technical upgrading, expansion of the number of services, enhancement of the standard of services, teachers’ in-service training, teachers’ premium, and students’ social support. There was an
‘Other’ category, where school heads could add other areas.
ANIKÓ FEHÉRVÁRI: Educational Effectiveness and Disadvantage
Table 1 | Regression model of mathematics and reading comprehension value added
Variable B Sig. R2
Mathematics Teachers’ continuing
training -55.4 .014 29.5
Reading
comprehension Teachers’ continuing
training -49.5 .007 29.2
CONCLUSION
In our study on educational effectiveness we presented a school category where the student population consists mainly of disadvantaged and Roma students. By linking the research and administrative databases we sought answers to the question what school factors influenced educational effectiveness. This was measured by value added calculated from the site-level National Assessment of Basic Competencies data.
Similarly to other effectiveness research, this study has not found a significant correlation between educational effectiveness and the school’s material assets.
The effect of is staff related factors is much stronger. For the category of schools explored in this study the absence of conditions is particularly important: lack of teachers with the required specialisation and absence of teaching assistants have a negative impact on value added. Scarcity of funds also has a negative effect which is experienced most sorely in teachers’ continuing training and the expansion of services. Value added is increased by the school’s intensive communication with local institutions and organisations active in other areas (concerning students), and participation in central development projects, particularly extracurricular activities, also has a positive effect.
Returning to international literature, our analyses highlighted professional development as the variable most closely correlated to educational effectiveness but the quality of the educational content also appeared through the positive effect on extracurricular activities. The school’s relations and its embeddedness in the local community emerge as a new factor possibly related to the fact that our study addressed the effectiveness of schools with a disadvantaged student population.
REFERENCES
Andor, Mihály – Liskó, Ilona (2000): Iskolaválasztás és mobilitás [School choice and mobility].
Iskolakultúra, Budapest.
Balázsi, Ildikó – Lak, Ágnes Rozina – Ostorics, László – Szabó, Lívia Dóra – Vadász, Csaba (2014):
Országos kompetenciamérés 2014 Országos jelentés [National Assessment of Basic Competencies 2014. National report]. Budapest: Educational Office
118
CURRICULUM, EFFECTIVENESS, EQUITY
Balázsi, Ildikó – Horváth, Zsuzsanna (2011): A közoktatás minősége és eredményessége [The quality and effectiveness of public education ]. In: Balázs, Éva – Kocsis, Mihály – Vágó, Irén (eds.): Jelentés a magyar közoktatásról 2010 [Report on Hungarian public education]. Budapest: HIERD
Benito, R. – Alegre, M.A. – Balletbò, I.G. (2014): School Segregation and Its Effects on Educational Equality and Efficiency in 16 OECD Comprehensive School Systems Comparative Education Review, Vol. 58. No. 1. pp. 104–134.
Coleman, J. S. – Campbell, E. Q. – Hobson, C. F. – McPartland, J. – Mood, A. M. – Weinfeld, F. D.
– York, R. L. (1966): Equality of educational opportunity. US Government Printing Office, Washington, DC.
Creemers, B.P.M − Kyriakides, L. and Sammons, P. (eds.) (2010): Methodological Advances in Educational Effectiveness Research. Routledge/Taylor & Francis, London, New York. http://
rufiismada.files.wordpress.com/2012/02/methodological_advances_in_educational_
effectiveness_research__quantitative_methodology_series_.pdf
De Maeyer, S. – van der Bergh, H. – Rymenans, R. – van Petegem, P. – Rijlaarsdam, G. (2010):
Effectiveness criteria in school effectiveness studies: Further research on the choice for a multivariate model. Educational Research Review No. 5. 81–96.
Fehérvári, Anikó (2015): A hátrányos helyzetű tanulók oktatásának változása, 2006–2014. In:
Fehérvári and Tomasz (ed.) (2015): Kudarcok és megoldások. Iskolai hátrányok, lemorzsolódás, problémakezelés. HIERD, pp. 31–52.
Fehérvári, Anikó – Tomasz, Gábor (eds.) (2015): Kudarcok és megoldások: Iskolai hátrányok, lemorzsolódás, problémakezelés [Failures and solutions: disadvantage attrition and problem management in schools].HIERD, 252 p.
Gyökös, Eleonóra – Szemerszki, Marianna (2014): Hol tart ma az oktatáseredményességi kutatás?
[State of the art of educational effectiveness research]. Új Pedagógiai Szemle, No. 1–2. pp. 43–64.
Hattie, J. A. C. (2009): Visible Learning: A synthesis of over 800 meta-analyses relating to achievement.
London: Routledge.
Hattie, J. A. C. – Anderman, E. (2013): International guide to student achievement. New York, NY:
Routledge
Liskó, Ilona – Fehérvári, Anikó. (2008): Hatásvizsgálat a HEFOP által támogatott integrációs program keretében szervezett pedagógus-továbbképzésekről [Impact analysis of teachers’ continuing training programmes provided in the context of integration programme supported by HRDOP], HIERD, 156 p. /Kutatás Közben 281./
Liskó, Ilona (2008): A továbbképzések hatása a pedagógusok szemléletére [The effect of research on teachers’ attitude]. Educatio No. 4. pp. 495–511.
Kertesi, Gábor – Kézdi, Gábor (2010): Iskolázatlan szülők gyermekei és roma fiatalok a közép-iskolában. Beszámoló az Educatio Életpálya-felvételének 2006 és 2009 közötti hullámaiból. In:
Kolosi and Tóth (ed.): Társadalmi Riport, 2010. Tárki, Budapest
Madaus, G. – Airasian, P. – Kellaghan, T. (1980): School effectiveness: a reassessment of the evidence.
McGraw-Hill, New York.
Reynolds, D. – Sammons, P. – De Fraine, B. – Townsend, T. – Van Damme, J. (2011): Educational effectiveness research (EER): a state of the art review. Paper presented to the annual meeting of the International Congress for School Effectiveness and Improvement, Cyprus.
Scheerens, J. – Glas, C. – Thomas, S. M. (2003): Educational Evaluation, Assessment and Monitoring.
A Systemic Approach. Lisse, Swets & Zeitlinger Publishers.
Scheerens, J. – Witziers, B. – Steen, R. (2013): A Meta-Analysis of School Effectiveness Studies. Revista de Educación, 361. Mayo - agosto 2013
Széll, Krisztián (2015): Iskolai eredményesség és tanári attitűdök [Educational effectiveness and teachers’ atttitudes]. In. Szemerszki (ed.): Effectiveness in education, Budapest: HIERD, pp. 25–51.
ANIKÓ FEHÉRVÁRI: Educational Effectiveness and Disadvantage
Reynolds, D. – Sammons, P. – Fraine B. D. – Townsend, T. – Damme, J. V. (2011): Educational Effectiveness Research (EER): A State of the Art Review. Paper presented at the annual meeting of the International Congress for School Effectiveness and Improvement, Cyprus http://www.icsei.
net/icsei2011/State_of_the_art/State_of_the_art_Session_A.pdf
Robinson, V. M. J. – Lloyd, C. A. – Rowe, K. J. (2008): The impact of leadership on student outcomes:
An analysis of the differential effects of leadership types, Educational Administration Quarterly, Vo. 44. No. 5. pp. 635–74.
Teddlie, C. – Reynolds, D. (2000): The International Handbook of School Effectiveness Research. New York: Falmer Press
Townsend, T. (1994): Goals for effective schools: The view from the field. In: School Effectiveness and School Improvement, Vo. 5. No. 2. pp. 127–148.
Varga, Júlia (2009): A tanárok elosztása a különböző szociokulturális hátterű tanulókat tanító iskolák között [Distribution of teachers in schools with student populations of different sociocultural backgrounds]. In: Fazekas, Károly (ed.): Oktatás és foglalkoztatás. Budapest: MTA KTI pp. 65–82.