Many of the data analyzed in this chapter are from DMQ 17, the penultimate version of the questionnaire. DMQ 17 had four age versions: infant, pre-school, elementary pre-school, and teen. The latter two age versions provided both adult ratings of the school-age child and forms for student-self ratings.
There were 45 Likert-type items each rated 1-5 (from not at all typical to very typical) and seven scales as follows:
Four scales for the instrumental or persistence aspects of mastery mo-tivations were:
1) Object-Oriented Persistence scale (called persistence at cogni-tive tasks for school-age children and teens; 9 items)
2) Gross Motor Persistence scale (8 items)
3) Social Persistence/Mastery Motivation with Adults scale (6 items)
4) Social Persistence/Mastery Motivation with Children scale (6 items)
Two scales for the affective aspects of mastery motivation were:
1) Mastery Pleasure scale, positive affect after finishing a task
(upset, avoid etc.) were scored as 5. The competence scale had 2 out of 5 items worded negatively and reverse coded.
More than 20,000 children from 6-month to 19-years of age were rated with DMQ 17. These include more than 1000 atypically developing children with a variety of delays and more than 500 children at risk for lower aca-demic achievement due to low socioeconomic status (SES). Geographically and linguistically, the children were very diverse. Participants included Eng-lish speakers from the United States, Canada, the UK, and Australia. Chi-nese speakers were from mainland China and Taiwan. In Hungary, more than 10,000, mostly typically developing, school-age children rated them-selves and/or were rated by their parents and teachers.
The Chinese version of DMQ 17 was translated by Ai-Wen Hwang in Tai-wan to use with infants and young preschoolers and by Jun Wang in main-land China to use with school-aged children. The Chinese-speaking re-searchers and the original developers of DMQ 17 – George A. Morgan, Karen Caplovitz Barrett, and Nancy Busch Rossnagel – went through multiple it-erations of translation, back-translation, pilot testing, and revisions to en-sure better the conceptual equivalence of the questionnaire items and the cultural and linguistic appropriateness of the Chinese version. Feedback from Chinese-speaking parent and child respondents was also solicited and incorporated during the development of the Chinese versions. For example,
“cause and effect activities” in English DMQ 17 did not have readily available Chinese translations and was hard for Chinese-speaking populations to un-derstand. The English-speaking and Chinese-speaking researchers dis-cussed the “cause and effect” phrase intensely, in order to reach mutual un-derstanding about the activities and to come up with appropriate transla-tions and clarifying examples for Chinese-speaking respondents. Similarly, gross motor persistence items concerning throwing and catching objects were easily relatable for English-speaking American respondents, consider-ing their familiarity with ball games like baseball and basketball. However, such ball games and corresponding physical skills were not as popular for Chinese-speaking respondents as for their English-speaking counterparts.
Thus, clarifying examples specifying the ball games were needed for Chi-nese-speaking respondents to make sense of the throwing and catching skills mentioned in the questionnaire items.
The Hungarian language version of DMQ 17 also went through similar processes of carefully translating and calibrating the expressions of both the questionnaire instructions and items. The first Hungarian versions of the Dimensions of Mastery Questionnaires were developed in the spring of 1999. Since then, the leading Hungarian-speaking researcher, Krisztián Józsa, and the DMQ’s original developer, George A. Morgan, have been col-laborating closely for decades to continuously refine the Hungarian versions
qualities, cross-national adaptation, and longitudinal application of DMQ 17.
As mentioned in Chapter 1, a number of journal articles, dissertations, and presentations included DMQ 17; several are noted in the reference list and many are cited in other chapters. These papers summarized evidence for reliability and validity, relationships to other variables, and also com-pared the three main cultures at similar ages and across ages. Although many theoretical and empirical studies were conducted on mastery motiva-tion using DMQ 17, the psychometric qualities of DMQ 17 were not fully evaluated in the initial studies. In particular, confirmatory factor analysis to carefully examine factorial validity and measurement invariance was only used with DMQ 17 in a few more recent studies. Confirmatory factor analysis (CFA) is a statistical technique to test the fit between hypothesized models and empirical data; it allows estimation of measurement errors to achieve a more precise estimation of factor loadings. Use of CFA informed the revision of DMQ 17 by enabling deletion of items with lower loadings. In addition, CFA conducted with multiple samples simultaneously can be used to check measurement invariance, the establishment of which ensures that compar-isons across groups with the same measure are meaningful (Schumacker &
Lomax, 2004). Only three studies to date have used CFA with DMQ 17, one was with preschool children (Hwang et al., 2017) and two were with school-age children (Józsa & Kis, 2016; Wang et al., 2014).
Structural Validity of a School-age Sample in Hungary
One of these studies, Józsa and Kis (2016), analyzed students’ self-ratings with CFA in a Hungarian school-age sample. The study verified the struc-tural validity of DMQ 17. However, the authors pointed out that the model fit indexes and the scale reliabilities could be improved by omitting some reversed items. However, the study did not cross-validate the equivalence of the DMQ across different age groups or cultural groups. The other two studies both used samples from Hungarian-, English- and Chinese-speaking samples, thus are described in detail in this chapter.
invariance study for the preschool version of DMQ 17. As DMQ was initially developed for mothers or caregivers to rate preschool children (Morgan et al., 1993), the preschool version of DMQ has the longest history and has in-fluenced the development of DMQ versions for other age groups and re-spondents. Psychometrically sophisticated examination like the Hwang et al. study (2017) is necessary to justify whether the scale items and underly-ing mastery constructs can be interpreted in a conceptually similar manner and be quantified and compared meaningfully across different groups of re-spondents. Specifically, the goals of the Hwang et al. study (2017) included 1) validating the hypothesized 5-factor structure (the four persistence scales and the mastery pleasure scale expected to underlie the items of DMQ 17 that were analyzed); 2) examining measurement invariance of parental rat-ings across English-speaking, Chinese-speaking, and Hungarian-speaking preschoolers; and 3) providing empirical support for revisions leading to DMQ 18.
A total of 1,582 English-speaking, Chinese-speaking, and Hungarian-speaking preschoolers children aged 24–72 months were rated by their par-ents with DMQ-17 preschool version. Chinese-speaking children (n = 389) were from Taiwan (the Taipei birth panel study, 2008; Hsieh et al., 2011).
English-speaking children (n = 353) were from the U.S. and Australia. The Hungarian-speaking children (n = 840) were from Hungary, providing a much larger sample size than the other two samples.
The four persistence and the mastery pleasure scales were used as in-dexes of mastery motivation in this preschool study. The General Compe-tence dimension was not included because it was not considered to be a measure of mastery motivation. The Negative Reactions to Failure scale was not included either, because this scale had inadequate internal consistency in DMQ 17, as noted above. Therefore, thirty-five items from DMQ 17 were used for the preschool measurement invariance study. The five dimensions examined include Object-Oriented Persistence (COP, 9 items), Gross Motor Persistence (GMP, 8 items), Social Persistence with Adults (SPA, 6 items), Social Persistence with Children (SPC, 6 items), and Mastery Pleasure (MP, 6 items).
To examine measurement invariance of the preschool data, the data from all the 1,582 children were randomly separated into two subsets: sample 1 (n = 791) and sample 2 (n = 791). The initial CFA model was explored with sample 1 to examine the factorial validity of a five factor model of DMQ 17 (i.e., COP, GMP, SPA, SPC, and MP) and to compare the goodness of fit be-tween a first order model and a second order model. The first-order CFA was estimated by allowing the five latent variables to be freely correlated.
The second–order CFA was a more parsimonious model with the five latent variables loaded onto one second-order factor. After identifying the best
fit-ting 5-factor structure model, the researchers used sample 2 to cross-vali-date the final model (Bollen, 1989). Then, samples 1 and 2 were merged for the examination of measurement invariance between samples 1 and 2, and among Chinese-, English-, and Hungarian-speaking groups.
First, based on lower loadings and poor fit with predicted factor structure (see also Chapter 5), 5 items with loadings lower than .45 were deleted, which included three COP items, one SPA item, and one MP item. Two out of these 5 items were negatively worded items that needed to be reverse-coded. Factor loadings, Cronbach's alphas, and composite reliabilities were all acceptable for each of the five scales. However, only three reversed items remained, across the 5 scales. Because of known problems in other samples with the reversed items (Jόzsa et al., 2014; Jόzsa & Morgan, 2017), these three items were omitted despite having loadings >.45. Thus, these items, together with the 5 items with low loadings were deleted, leaving 27 items to be used in testing the final confirmatory model with sample 2. The eight omitted items are presented in Table 2.1. Discriminant validity with boot-strapping suggested that the five factors were discriminative between each other.
The second order model, which modeled the 5 domain-specific mastery dimensions under a broader mastery motivation construct, fit the data as well as the first order model. Because the second order factor structure is more closely aligned with the theoretical conceptualization of mastery mo-tivation, it was selected for the remaining analyses. Mastery motivation was modelled as a latent variable which is not observable directly but can be in-ferred from the shared variance (the conceptual and empirical overlap) of the five mastery motivation dimensions, COP, GMP, SPA, SPC, and MP.
Each of these five dimensions of mastery are also latent variables themselves which cannot be observed directly but can be inferred from the shared var-iance of a subset of the 27 items. Besides the shared varvar-iances, each of the 27 items and the five mastery dimensions were allowed to have measure-ment errors (e), which were also modeled in the CFA. Such a modeling tech-nique allows for a more accurate estimation of the latent constructs.
Table 2.1. DMQ 17 Preschool Items Deleted Based on Fit with Expected Structure Using Confirmatory Factor Analysis
DMQ scales/items Standardized
loading Object Oriented Persistence (COP)
7 Likes to try hard things instead of easy ones. .435L 9 If a toy or task is hard to do, stops trying after a short time. .460R 17 Explores all parts of an object or toy with many parts. .316L 24 Tries hard to do cause and effect toys such as a busy box. .440L Gross Motor Persistence (GMP)
3 Gives up if he or she cannot do physical skills well. .566R Social Persistence with Adults (SPA)
33 Gives up quickly when playing with adults. .319LR Social Persistence with Children (SPC)
39 Avoids getting involved with other children. .555R Mastery Pleasure (MP)
11 Does not smile after he or she makes something happen. .322LR Note. L: Loadings < 0.45; R: Reversed item.
With the second order CFA model, measurement invariance was exam-ined between samples 1 and 2 and among the Chinese-, English-, and Hun-garian-speaking groups. The factor loadings, structure weights, and struc-tural covariances of the same items or constructs were progressively con-strained, enabling them to be invariant across the English, Chinese, and Hungarian language samples. In other words, each successive model in-cluded the previous model’s restrictions plus additional constraints and served as the comparison standard for the subsequent model until an invar-iant structure fits data from all samples. Based on this established measure-ment invariance in structure, latent mean differences could be and were ex-amined across the three groups. There were no differences between the three language groups except that the Chinese-speaking preschool children were rated lower than the other two groups on gross motor persistence by their parents.