Systematic literature review approach

A systematic review is a research method that is undertaken to review research literature, using systematic and rigorous methods (Gough, Oliver, & Thomas, 2012). For the present study, systematic literature has been defined as:

A review of a clearly formulated question that uses systematic and explicit methods to identify, select, and critically appraise relevant research, and to collect and analyse data from the studies that are included in the review. Statistical methods (meta-analysis) may or may not be used to analyse and summarise the results of the included studies. (The Cochrane Collaboration, 2005).

Systematic reviews are often referred to as ‘original empirical research’ because they review primary data (Aveyard & Sharp, 2011). They can be quantitative using statistical methods to combine data from multiple studies (meta-analysis), or qualitative (synthesis) in which findings are summarized and synthetized using qualitative methodology (Gough et. al., 2012). There are two broad types of qualitative synthesis. First, integrated reviews, aggregate or summarise data using themes, while interpretative reviews involve interpreting the data using an inductive approach from which new conceptual understandings can emerge (Gough & Thomas, 2012).

In Study 1 a qualitative systematic literature review, more specifically a thematic synthesis, was chosen to describe, appraise and synthesize the current empirical research on in-service K-12 teachers’ beliefs about creativity and its nurture, given its methodological strength as a means of establishing a comprehensive and reliable evidence base (Gough et al., 2012, p. 2). To ensure that the review was systematic, Study 1 was guided by the Preferred Items for Systematic Reviews and MetaAnalysis (PRISMA) statement (Moher, Liberati, Tetzlaff, Altman, & ThePrismaGroup, 2009), and included the following steps: (1) defining relevant studies and establishing inclusion/exclusion criteria; (2) developing the search strategy; (3) identifying potential studies through searching and screening; (4) describing and appraising included studies;

(5) analysing and synthesizing findings.

Eligibility criteria

Eligibility criteria for the present review were the following:

• Topic of research: studies designed to describe and explore teachers’ beliefs about creativity, creative students, and creative pedagogy (creative teaching, teaching for creativity, creative learning).

• Type of research: full primary reports of empirical research (qualitative, quantitative, and mixed method).

• Study population and settings: research whose primary participants were in-service teachers active in K-12 education settings.

• Date of publication: studies published between January 2010 and December 2015

• Language of publication: studies written in English.

• Transparency: studies which explicitly described the theory, methodology, and data on which conclusions rest.

• Reliability/validity: studies whose findings are valid and reliable, considering the type of study.

Exclusion criteria were the following:

• Studies that contained incidental data on K-12 teachers’ beliefs about creativity.

• Studies that collected data in gifted, early childhood and tertiary education settings.

• Studies whose findings on K-12 teachers’ beliefs about creativity could not be separated from those of other populations’, such as teachers active in gifted, early-childhood, tertiary education settings, or students, parents, head teachers, and other stakeholders.

Search strategy

To locate as much of the potentially relevant literature on teachers’ beliefs about creativity published between 2010-2015 as possible, I searched different sources between February and April 2016, presented in detail below.

Electronic databases searched in this review included those relevant to education, educational psychology and psychology: ProQuest ERIC, EBSCO PsychInfo. In addition, I searched ProQuest Dissertation and Theses Global for dissertations on the topic in English accepted for higher degrees (PhD, EdD). Search terms developed for database search included belief terms, creativity terms and teacher terms, which were established upon a preliminary review of the literature and on the previous synthesis of the research on teachers’ conceptualizations of creativity between 1991-2010 by Andiliou and Murphy (2010). Pilot searches on single and combined terms were also carried out before deciding on a final list of keywords, which are presented in Figure 7.

To avoid limitations of using pre-determined search terms and controlled vocabulary (Brunton, Stansfield, & Thomas, 2012), hand searches were made for the period between 2010-2015 of the following key journals: Creativity Research Journal, Journal of Creative Behavior, Journal of Psychology Aesthetics, Creativity and the Arts, Thinking Skills and Creativity, International Journal of Creativity and Problem Solving.

Further searches included the forward reference list checking of key articles on the topic using Google Scholar, i.e. those published by Aljughaiman and Mowrer-Reynolds (2005), Andiliou and Murphy (2010), Fryer and Collings (1991), Runco and Johnson (2002) and Westby and Dawson (1995) as well as the reference list checking of studies included in the present review. Finally, I searched Google for further hits and asked personal and professional contacts.

Figure 7. Keywords identified for database search in Study 1

Study selection

The database search delivered 2,600 references. After removing 387 duplicates, titles and abstracts of the remaining 2,213 articles were divided between my advisor and me, and screened individually for preselection purposes using the inclusion and exclusion criteria. A random sample of 10% percent of the articles were chosen and screened by both of us. Interrater reliability was calculated for the sample showing almost perfect agreement (Kappa= 0.951). Screening thus produced 61 studies, the full texts of which were acquired to be checked applying the inclusion/exclusion criteria. Discrepancies were solved by discussion until 100% agreement was reached.

Of the 61 studies resulting from the database search, 12 have been excluded based on the inclusion/exclusion criteria. Hand searching produced 28 additional studies, of which a further seven have been excluded with reason. Finally, the search strategy applied yielded a sample of n=70 studies which were then judged for their quality and relevance.

Quality and relevance appraisal

After judging pre-selected studies according to the inclusion/exclusion criteria, the remaining 70 studies were assessed for their quality and relevance based on the Weight of Evidence (WoE) framework outlined by Gough (2007). Following this framework, studies were appraised in relation to three key areas: methodological quality (WoE A), methodological relevance and topic relevance (WoE C). For more details, see the scoring sheet produced to aid the appraisal process presented in Appendix A. For a graphic overview of the selection procedure see Figure 8.

Figure 8. PRISMA Flow Diagram of study selection for the systematic review in Study 1

The appraisal of studies was undertaken by me while another educational researcher with a PhD assessed a random selection of studies (25%, n=15 of the 60 studies included in the final pool). The two judges reached full agreement on the quality and relevance ratings awarded to the sample.

Studies considered inadequate in any area, as well as studies rated low in the methodological quality area were excluded from the review. The appraisal of pre-selected studies, thus resulted in the exclusion of further 18, which based on their weight of evidence were considered inadequate for the present review: seven studies did not meet the quality criteria, whereas eleven did not satisfy the topic relevance criteria. For the rest of the studies the results of the methodological quality, methodological relevance and topic relevance appraisal were combined and given an overall weight of evidence of high, medium or low. The results of the quality appraisal of the included studies are provided in Appendix A.

The search and appraisal strategy applied yielded a final sample of n=53 studies as presented by Figure 8.

Data extraction and analyses

Data extraction from the 53 selected studies was conducted using a template which recorded key information about the sample: (1) authors, (2) year of publication, (3) type of publication, (4) study purpose, (5) research questions/hypotheses, (4) type of view examined, (3) view topic, (4) country, (5) grade level, (6) subject area, (7) sample characteristics, (8) research design, (9) research instruments, (10) data analysis procedures, (11) major findings, (12) summary of major findings, (14) creativity definitions.

Surface characteristics of the reviewed studies were analysed using descriptive statistics. For the synthesis of findings in relation to the first research question, which focused on the characteristics of teachers’ beliefs about creativity, we applied mixed-coding strategy with some pre-defined concepts (Oliver & Sutcliffe, 2012) building on Andiliou and Murphy’s (2010) conceptual framework on the topic. For the questions concerning teachers’ beliefs and perceptions in relation to classroom practice and the factors that might influence teacher’ views about creativity, data was open-coded (Oliver

& Sutcliffe, 2012). Coding in both cases was carried out in NVivo 11 for Mac, while themes as well as the decisions taken regarding these were discussed with my advisor.