3 Research Methods and Strategies
3.3 Data Collection Strategies and the Development of Research Tools: The Think-Aloud
3.3.1 The Think-Aloud Data Collection Approach and its Components
Leighton (2017) praises the think-aloud data collection strategy because of its potential to yield “a wide range of responses leading to the understanding of the boundaries, parameters, nature and character of the phenomena” (p.5). In my approach to the phenomena of the organization and dynamics of the MML, I decided to develop a think-aloud activity that made possible:
Figure 9. Research Tools and Their Chronological Order of Development in the Study.
The arrow represents the sequence in which the research tools were developed. The development of the questionnaire started almost concomitantly with the development of the think-aloud activity and was finalized some time after the development of the think-aloud activity ended. The place of the questionairre within the sequence is indicated by the dotted lines.
(1) the observation of the participants “practices, perspectives, and processes”
(Leighton, 2017, p. 5) during a meaning-making task requiring the use of the L3 English,
(2) keeping the participants’ language activation at a minimum so as to support the observation of which language(s) played a primary role in the process of meaning making, and
(3) the observation of whether and how languages interacted, hindered or helped each other in the process of meaning making.
The think-aloud activity that resulted asked the participants to examine the pictures and compose a narrative in English, their L3. They had to write down the L3 narrative. Figure 10 shows the main elements of the think aloud activity. Thinking aloud meant that the participants talked aloud while they were composing the text and verbalised whatever thoughts came to their mind. The task did not have a time limit. Each participant’s task-solving process was recorded and transcribed for analysis.
The think-aloud data collection approach employed in the study is a compound of several elements each with its own theoretical underpinning. The following sections are dedicated to the discussion of the elements of thinking- aloud while solving a problem: what it means to think aloud, the cognitive processes involved in meaning making of pictures, the cognitive processes involved in composing a narrative in a non-native language, and the Figure 10. The Elements of the Think-Aloud Data Collection Process.
advantages writing by hand has to collecting verbal data in alouds. Because the think-aloud data collection strategy is evaluated based on the characteristic of adequacy, that is its potential to elicit verbalizations, the cognitive load it places on the problem-solver, and the expertise it requires (Beilock & Carr, 2001; Beilock, Kulp, Holt, & Carr, 2004; Bickart &
Felcher, 1996; Charters, 2003; Leighton, 2017), the elements are discussed with a special focus on this characteristic.
3.3.1.1 Thinking Aloud
Thinking aloud, or concurrent verbal report, is a data collection approach designed to observe cognitive processing within the limits of working memory (Alderson-Day &
Fernyhough, 2015; Fernyhough, 2016). Cognitive processes and the mental lexicon itself cannot be explored directly, so we can only draw inferences and arrive at “piecemeal approximations” (Wimsatt, 2007) in the models and theories we put forward as a result of our explorations.
The think-aloud data collection strategy offers researchers special lenses through which to look at cognitive processes and the mental lexicon and supports the process of
“making claims about unobservable psychological phenomena” (Leighton, 2017, p. 2), claims which, in turn, can contribute to formulating conclusions or hypotheses. To understand how the think-aloud procedure works, it is useful to liken it to Brunswik’s (1952, 1966) lens model.
The lens model postulates that we examine proximal cues (i.e., available information to our senses) to infer distal properties that are not directly accessible. From the standpoint of the think-aloud procedure, the most useful property of the lens model is the proximal cue-lenses-distal property triad because it allows the understanding of the role the think-aloud procedure plays in the processes of examination and drawing inferences. In the case of the exploration of the mental lexicon, researchers use the think-aloud procedure as lenses to examine proximal cues (i.e., verbal protocols) in order to infer regularities about the organization and functioning of the mental lexicon. We are aware of the caveat that none of the collected information perfectly depicts the mental lexicon. However, we assume that there should be some relation between how words are accessed and retrieved and the organization and the functioning of the mental lexicon. Thus, the mental lexicon, its organization and ways of working are distal issues that we infer from the voiced inner speech of the participants, that is the proximal cues.
One of the earliest studies taking the position that thinking aloud is in fact voiced inner speech is that of Benjafield (1969), who argued that “thinking aloud makes overt the normally covert process of inner speech” (p.83). Later, Shergill et al. (2001) demonstrated that during overt and inner speech the same brain areas are activated, and they concluded that inner
speech is overt speech without sounds. Alderson-Day and Fernyhough (2015) argued that our self-talk and vocalized self-guidance is exteriorized inner speech. These observations bear importance to the present discussion first because they prove that the language we use in our wilful verbal thought is the same language present in our exteriorized speech (Peronne-Bertolotti et al., 2014). Furthermore, they also demonstrate that thinking aloud our inner speech which surfaces during reasoning processes requires no more cognitive effort than speech itself (Centeno-Cortés & Jiménez Jiménez, 2004). Larrain and Haye (2012) remarked, regarding the discursive nature of thinking aloud, that it is a special group of discursive practices ranging from “syntactically well-organized self-talk (…) to processes that are on the border of language because of a weak or very instable syntactic organization” (p.
3). It is noteworthy that the discourse which surfaces as think-aloud, the verbal report, has several specific characteristics that make it very different from the discourse of social speech (Winsler et al., 2005).
Four main characteristics have to be mentioned in this regard. First, think-aloud discourse is midway between monologic and dialogic speech. Think-aloud verbal reports turn overt the individual’s covert inner speech, which is primarily monologic (Vygotsky, 1978).
However, being directed to the self, for instance in terms of its self-motivating or self-directing components, think-aloud verbal reports are also dialogic (Frawley, 1997; Morin, Duhnych, &
Racy, 2018; Plato, n.d./1892). Second, think-aloud verbal reports are spontaneous. They are delivered in response to a task that the individual engages in. The fact that they are spontaneous allows the researcher to look at how the process of understanding and solving the task take place in real time. Third, think-aloud verbal reports typically contain repetitions, hesitations, pauses, false starts, reformulations and digressions, all these in fact mirroring how units of information are attended in the individual’s working memory (Charters, 2003;
DeSouza et al., 2008; Holsánová, 2008; Jiménez Jiménez, 2015; Leighton, 2017). The last characteristic of think-aloud verbal reports is that they contain fragmented discourse with
“spurt-like portions” (Holsánová, 2008, p. 1) and often elliptic sentences (Larrain & Haye, 2012).
In several think-aloud research studies, the steps taken by the participants engaged in solving the think-aloud task were contrasted with the steps contained by already existing cognitive models. Researchers, such as Newell and Simon (1972), Ericsson and Simon (1984), and Van Someren et al. (1994) put forward a predicted behaviour and identified those aspects of the participants’ behaviour that matched and diverged from the cognitive model.
In the present study, I adopted a multi-grounded theoretical perspective and developed the think-aloud activity starting from a broad model incorporating theories on general problem solving (Simon, 1978), theories on meaning making (Kurzman, 2008; Zittoun & Brinkmann,
2012), theories on picture comprehension and pictorial series comprehension (Biederman, 1987; Eisner, 1985; Entman, 1993; Graesser, Singer, & Trabasso, 1994; Holsánová, 2008;
Kahneman & Tversky, 2000; McCloud, 1994; Neisser, 1976; Petterson, 1989; Salomon, 1984;
Zacks, Tversky, and Iyer, 2001), and theories on transposing pictorial representations into words (Aktulum, 2017; Dusi, 2015; Jakobson, 1959; Rojo Lopéz, 2015). All these theories are briefly presented in Section 3.3.1.2. As a second step, following the multi-grounded research paradigm, I expanded the initial broad model incrementally by including the steps of how the participants carried out the linguistic task, accessed their mental lexicons, and relied on their languages. I started from the codes of the first think-aloud protocols and assembled an incipient comparative framework, which I developed along the study by expanding on iterating themes. It has to be mentioned here that, to my knowledge, no cognitive representation which models word-search in tri- or multilinguals’ composition processes has been put forward yet.
The various aspects of the coding I performed will be detailed in the section of the thesis dedicated to data analysis.
3.3.1.2 Making Meaning of Series of Wordless Pictures: A Special Type of Problem Solving and Elicitation Tool in the Study of the MML
Among the tasks that can lead to abundant verbal reports as data are those which require the participants to put to use both their visual interpretive and verbal productive skills (Berman & Slobin, 1994; Holsánová, 2008). One such task is the meaning-making of graphic narratives or pictures arranged in a logico-temporal sequence to tell a story (Eisner, 1985).
Meaning-making is “the process by which people interpret situations, events, objects, or discourses in the light of their previous knowledge and experience” (Zittoun & Brinkmann, 2012, p. 1809). Tasks requiring the meaning making of graphic narratives have been used in think-aloud processes both because they are considered problem-solving tasks and because they make possible the scrutiny of the mental activity taking place in the verbal working memory (Charters, 2003; Ericsson & Simon, 1984; Van Someren et al., 1994). Because of these properties, graphic narratives can be considered valuable tools in the examination of the MML in terms of linguistic availability for picture interpretation in a non-native language (Berman & Slobin, 1994; Wekesa, 2012).
The adequacy of graphic narratives as tasks in the think-aloud data collection approach is determined primarily by their capacity to make visible the problem space, that is, the solver’s paths taken from the problem’s initial state to the goal state in transposing the non-verbal content of pictures into non-verbal content (Bassok & Novick, 2012; Newell & Simon, 1972;
Reed, 2015, 2016; Reed & Abramson, 1976; Simon, 1978). Simon (1978) put forward the following components of the problem space:
1. A series of knowledge generating states, which lead from the problem’s initial state to its solution state.
2. A series of operators, or movements, which connect the knowledge generating states and are oriented towards the solution state.
3. A series of restrictions which allow the solver to move within the limits of the task.
4. Knowledge about each knowledge state, and the steps that have been taken before and after a particular state.
The components of Simon’s (1978) problem space can be mapped onto the task of meaning-making in a non-native language of a graphic narrative. In the resulting arrangement, the initial state of the problem is the series of pictures that have to be interpreted and re-composed in a non-native verbal language. The solution state is the story created by the participant based on the visual content. To complete the task, the participant has to both generate knowledge and transform knowledge from the visual modality into the verbal (Petterson, 1989). The steps he takes are incremental and build upon one another.
Even backtracking, a problem solving method pertaining to computational problem solving and consisting of considering several possible solutions and choosing the viable one can be identified in the process of meaning-making of graphic narratives in a non-native language.
This happens on two levels. The first is the level of choosing from among the words deemed by the participant fitting in the context of his narrative. To build up the story, he has to choose from among several possible words and expressions. The second level is picture-comprehension, where the participant chooses the visual information to be transposed in words and what to leave out (Entman, 1993; Kahneman & Tversky, 2000), and the order in which these pieces of information will be included in the story (Holsánová, 2008).
The operator-element in the task of meaning-making of a graphic narrative in a non-native language is the transposition of as much pictorial information as possible into the target verbal language of the task. The operators are subject to the constraints imposed by the general rules of story structure and coherence. The participant, in the possession of these knowledge frames, knows at any stage of the task-solving how he got there and what the following steps to be taken are.
Another requirement to be met by the tasks employed in think-aloud processes is that they have to involve genuine instances of problem solving (Ericsson & Simon, 1984;
Leighton, 2017; Van Someren et al., 1994). Problem solving in the case of graphic-narrative transposition into a non-native verbal language requires making meaning of pictorial information and translating it into words. It has been argued that understanding pictorial messages takes place at high cognitive levels and requires considerable mental effort (Biederman, 1987; Petterson, 1989; Salomon, 1984; Zacks et al., 2001).
The process is creative, incremental and idiosyncratic to each individual and to each situation (Kurzman, 2008). Transposing pictorial representations into words can be considered a “goal-driven process of finding a solution to a complex state of affairs”
(Leighton, 2017, p. 22) given that making meaning of pictorial representations is intersemiotic translation (Aktulum, 2017; Dusi, 2015; Jakobson, 1959; Rojo, 2015) and possesses the whole array of problem-solving characteristics particular to translation processes.
Understanding sequences of wordless picture series and transposing their content into words poses the challenge of linking the seemingly isolated picture panels to make them fit together into a cohesive story. In most Western wordless picture series, the events of the story unfold in time from left to right. The framed panels capture apparently isolated moments of time and space and, surprisingly, it is the gap between the panels that glues the series of panels together (Eisner, 1985; McCloud, 1994). The reader of wordless picture series has to
“connect these moments and mentally construct a continuous, unified reality” (McCloud, 1994, p. 67). Two distinct mental processes contribute to the success of this activity. First, as a result of our event-structure perception, we are able to “identify the parts of an event and their relationships” (Zacks et al., 2001, p. 29) A second mental process, called closure, assists us in “completing that which is incomplete” (McCloud, 1994, p. 63), supplying the missing information by drawing on our existing knowledge structures (Petterson, 1989;
Salomon, 1984). This is the point when we make inferences, or read between the panels (Graesser et al., 1994). Figure 11 contains a sequence of three picture panels and the description of a possible meaning making of what the pictures communicate.
Figure11. Sequence of Three Wordless Picture Panels.
The theme of this sort wordless story is revenge. In the first panel, we see a client who is ready to leave–he has his coat on, and is taking his hat–and a waiter who is not content with the tip he has been left. Looking at the second panel, we can infer that the waiter has turned from the table–as we see him heading towards the exit–but we do not know whether he took the coin left as a tip, or he left it under the plate. The second panel leaves us also in suspense as to what has happened to the client. These are all points where our imagination and creativity are to supply the missing information drawing on our past experiences. Several possibilities exist and all are viable moves towards the solution state of the meaning making, placed in a very conspicous place. Then, again between the second and the third panel, we
The complexity of the processes discussed makes it obvious that meaning making of wordless picture series is more than just supplying the words for the pictures. In interpreting pictorial representations, readers engage in a creative process where non-verbal and verbal mental representations interact (Neisser, 1976). In the light of the above discussion, it can be argued that individuals face an increased challenge when, in making meaning of wordless picture series, they have to come up with lexical units from a non-native language. The challenge of the task is even more increased when the non-native-language lexical units do not belong to the individual’s frequent lexis store (Bell, 1991; Gile, 1995). This is the reason why making use of an L3 in the intersemiotic translation of wordless pictures into words appears to be a promising strategy to study the MML by identifying how L3 lexical units and lexical units from the other languages participate in the construction of an L3 text elicited by pictures (Cavanagh, 2011).
The wordless picture series used as the elicitation material in this research study was Tomie DePaola’s (1978) Pancakes for Breakfast (see Appendix A). In adopting this particular picture series, my choice was based on Labov's (1972) story-structure theory, the theory of scene perception formulated by Henderson and Ferreira (2004), and the cognitive load theory in information processing proposed by Sweller (1988) and Hashtroudi, Mutter, Cole and Green (1984). As verbalising thoughts in a think-aloud task has been shown to require extra mental effort in certain situations (Hertzum, Hansen, & Sonderstrup-Andersen, 2009;
Leow & Morgan-Short, 2004; Sweller, Ayres, & Kalyuga, 2011), it was important to build the task around a picture series which would not heighten unnecessarily the participants’ mental effort yet at the same time contain some less familiar elements for the participants to make the process of search for lexical units visible to the researcher (Cavanagh, 2011).
DePaola’s (1978) wordless story consists of a chain of humorous events and possesses Labov’s (1978) six constituents of well-structured narratives: abstract, orientation, complicating action, evaluation, resolution, and coda. DePaola (1978) created suspense by intensifying the complicating-action and evaluation constituents of the story by using a repetitive problem-solution pattern, thus holding off the resolution. These movements make the picture story enjoyable and contribute to the think-aloud task’s ecological validity (Cohen et al., 2007; Libben & Jarema, 2002). The series of events of the story is also easy to follow, thus it adds a minimal load to the general cognitive load of the think-aloud task (Wernaart, 2012).
Cognitive load is also influenced by the perceptual processing of the visual input (Henderson & Ferreira, 2004; Henderson & Hollingworth, 1999; Wernaart, 2012), or scene perception. In terms of scene perception, DePaola’s (1978) wordless picture story has the characteristics of real-world scenes listed by Henderson and Ferreira (2004), which facilitate reader interpretation, and it increases the think-aloud task’s ecological validity. Research
showed that scenes that depict situations and environments typically encountered in our daily life are perceived and understood without demanding cognitive investment (Henderson
& Ferreira, 2004).
DePaola’s (1978) Pancakes for Breakfast presents an old lady’s adventures while she tries to prepare pancakes and encounters repeated difficulties. The author used colour- rendered line drawings to depict real-world agents (people and animals), real-world objects, natural environments, and mundane activities. All the depicted elements are nameable. The pictorial information present in the wordless story is the kind of information normally available in our everyday surroundings.
3.3.1.3 Narrative Story Composition
Starting from the wordless series of pictures, the participants had to compose a narrative story in their L3, English; that is, they had to produce an intersemiotic translation in a non-native language. The task was designed to elicit oral protocols which would make visible how the multilingual participants employed the target language of the task and possible other languages in the process of intersemiotic translation. One of my main assumptions was that the examination of this process would allow for drawing inferences about the organization and dynamics of the MML, and this in turn would make possible the kind of abductive reasoning which leads to hypothesis formulation.
Choosing to ask the participants to compose a narrative, and not another type of composition, was another strategic step in devising the think-aloud task. The narrative structure is one of the earliest generic forms we come to know as children (Hardy, 1974;
McCabe & Bliss, 2003; Minami, 2001; Tancz, 2009). Furthermore, the narrative is one of the commonest generic forms in educational settings (Donovan & Smolkin, 2006; Kress, 1982).
Van Peer and Chatman (2001) defined narratives as “texts about events structured in time (…), about agents who act in real or fictional worlds, responding to their inner drives as well as to external circumstances.” (p. 2) Narratives are inherent to human culture (Dodds, 2013;
Keszeg, 2002) and are part of the shared knowledge pertaining to both culturally bound schemas and universal constructs that make possible cross-cultural communication (Nishida, 1999). Based on these characteristics, I reasoned that incorporating in the think-aloud task the narrative as the text-type to be produced by the multilingual participants would be advantageous because it would not heighten the cognitive load of the meaning-making process. The research participants involved in the study were young multilinguals enrolled in secondary education, reasonably familiar with the narrative genre.
3.3.1.4 Writing
The last methodological consideration relating to the design of the think-aloud data collection strategy relates to the forms in which the participants had to produce the L3 narrative text. The task of producing the L3 narrative composition included the requirement to verbalize and to write the text down in longhand. The decision to ask the participants to write down the story they composed was primarily based on the assumption that writing would slow down the composition process and thus the whole idea and word-generating processes would become more visible. Additionally, the rationale for the use the oral production of idea generation followed by writing, rather than the use of only oral production or only writing, rested on cognitive research evidence for the benefits of using the inner voice or thinking aloud during the process of text composition (Berninger, 2009; Berninger & Chanquoy, 2012;
Chenoweth & Hayes, 2003; Davidson & Berninger, 2016). Wright (2018), comparing young story-tellers’ composing processes in the oral and written modes, found that her participants produced more words in the oral mode compared to the written mode, but the complexity of the language they used was very similar in the two modes. Neither at the sentence level nor at the discourse level of the compositions did she find statistically significant differences between the oral and the written modes. More recently, McAndrews (2020) has suggested that the oral and written production of narratives can be employed interchangeably with the aim of improving young learners’ oral language complexity (p. 236).
In the present research, the process of composing the narrative text was also supported by requiring the participants to use the traditional pen-on-paper writing mode instead of the computer-keyboard assisted writing. This decision was based on time-tested writing-skill development theory and the recent advances made by cognitive psychology research highlighting the roles handwriting plays in both creative processes and writing-research paradigms. Hajnal (1998) and Berninger, Abbott, Augsburger and Garcia (2009) pointed out that writing, contrary to oral delivery, forces the author to approach the subject more thoughtfully and to arrange ideas. These characteristics of writing contribute to the slowing down of the process of idea generation, and make possible its monitoring during a research process. Benczik (2006) highlighted another advantage of writing, namely that having to write down what is being composed disencumbers working memory because writers do not have to keep in mind the whole course of thought. What has been already written down acts as both as a reminder and a springboard for inspiration. More recent cognitive research on process writing showed that handwriting has special powers to mobilize creativity and improve the content of the piece of writing to be created (Berninger, 2012; Garibaldi & Harralson, 2017). In line with these methodological considerations, longhand writing was an element of the think-aloud protocol with a double aim. First, it made
the process of idea and word generation required by the building up of the L3 text available to observation. Second, it decreased the cognitive load on working memory during idea generation in the completion of a non-native linguistic task. At this point, it has to be highlighted that writing was only a research strategy, as described before, and not a target by itself. As a consequence, the written products of the think-loud task were not taken into consideration in data analysis. Data analysis of the think-aloud task focused on the verbal protocols collected during the processes of generating ideas, setting goals, and turning ideas into a cursive, logically coherent text, and it looked at how lexical units were accessed and chosen to build up the L3 narrative.
The think-aloud task sheet used in the study is reproduced in Appendix B. The instructions were given in English so as to set the language of the task and keep the activation of any other languages of the participants’ MML to a minimum (Bialystok, 2011;
Green, 1998; Poarch & Van Hell, 2012). The think-aloud task was not time-bound so as to alleviate the tension created by having to verbalize thoughts and to reduce the possibility of obstructive states such as tip-of-the-tongue, defined as “the feeling that a known word will be recalled even though it is not accessible immediately” (Schwartz, 2006, p. 149), and choking under time-pressure, defined as “performing more poorly than expected given one’s level of skill (…) in situations fraught with performance pressure” (Beilock & Carr, 2001, p. 701).
The design process of the task to use with the think-aloud data collection approach included a pilot-phase in which three L1 Polish-L2 Hungarian-L3 English multilingual secondary school students aged 16 built up an L3 narrative based on the picture series. After task completion, they were asked to talk about issues of building the text, word-search, and the extra task of saying aloud what went through their minds. The information resulting from the pilot-phase was used to refine the phrasing of the task instruction and to reduce its length.
The analysis of the verbal protocols resulting from the meaning-making process was performed based on the GTM paradigm, and it is discussed below in the section dedicated to data analysis.