The idea of globalization seems to fail to infect the field of second language acquisition where in the last ten years more and more emphasis has been placed on the traits in which language learners differ from each other. The triumph of the trend of individual differences is heightened by the fact that it had to overrule the several-decade-old tradition of universality, which was very convenient both for theorists and language teachers. This approach stated that the process of acquiring a second language was alike for all learners, who tended to struggle with similar difficulties and make similar mistakes. In contrast the study of individual differences emphasizes the variations language learners show both in instructed and naturalistic contexts. These differences can be put into three larger categories, namely affective, personality related and cognitive factors (Gardner 1985). Motivation, language learning anxiety and self-confidence belong to the affective category. Extraversion, conscientiousness and emotional stability are examples of personality related traits, whereas aptitude, intelligence and working memory capacity make up cognitive variables. In what follows I am going to examine one of the constructs in this latter group, aptitude.
The present article aims to compare findings of two case studies (Hild 2007; Hild in press) investigating the response validity of a recently developed aptitude test (MENYÉT, Ottó 1996) for Hungarian learners with the help of think-aloud protocol.
The examinations focused on the second component of MENYÉT: Language Analysis.
In the first part of the present study I provide a short overview on aptitude and aptitude research and introduce the Hungarian Language Learning Aptitude Test, MENYÉT.
Then, I describe the framework and the objectives of the empirical study. I also give detailed accounts of the participants, the instruments I used, the research setting and the results. Following the conclusion I put forward recommendations as to what issues should be considered prior to conducting a similar research.
Foreign language learning aptitude
Language learning aptitude belongs to the cognitive factors of individual differences. It can predict the degree of success in foreign language learning in both instructed and non-instructed contexts (Graaf 1997; Reves 1983 cited in Dörnyei-Skehan 2003: 10;
Robinson 1995), although experiments demonstrated that aptitude difference was less sensitive in naturalistic than in formal language learning contexts (Robinson 2002). It
154
is also considered to be independent of intelligence (Sparks-Ganschow 2001), and fairly stable (Pinker 1994; Skehan 1991, 1998). Some experts nevertheless hold a more careful view and claim that there is not enough evidence to prove its stability (Skehan 2002). It is also essential to note that language learning aptitude cannot foretell whether an individual can or cannot learn a foreign language. It can only determine the rate of language learning (Carroll-Sapon 1959).
Aptitude is made up of various independent abilities. Through factor analytic studies J. B. Carroll identified four components that were then considered the most relevant abilities constituting foreign language aptitude. These are (1) phonetic coding ability, (2) grammatical sensitivity, (3) inductive language learning ability, and (4) rote learning ability (Carroll 1981). Out of the four inductive language learning ability is the focus of the present study. This component refers to the ability to infer linguistic forms, patterns and rules on the basis of language materials, and then to produce new sentences. Grammatical sensitivity and inductive language learning ability are often considered very similar. However, the former requires good analytical skills, while the latter expects the individual to extrapolate and reason (Skehan 1989). Based on this model of foreign language aptitude Carroll and Sapon (1959) developed the Modern Language Aptitude Test (MLAT).
The Hungarian language aptitude test (MENYÉT)
The developer of MENYÉT, Ottó István, relied on Carroll’s four component model of aptitude. Consequently, the test consists of four subtests (Ottó 1996). The first one is Hidden Sounds (Rejtőző hangok), which measures phonetic coding ability. The second one, Language Analysis (Nyelvi elemzés), measures inductive language learning ability, and requires the students to choose the correct artificial language translation of Hungarian sentences from four alternatives, using a group of sample sentences and words written in this language and their Hungarian equivalents. Subtest 3, Words in Sentences (Szavak szerepe a mondatban), is designed to measure grammatical sensitivity. The students are provided with one key sentence in which one word is underlined and printed in capital letters. They are given a second sentence in which five words are underlined and they have to choose from these five alternatives the one that has the same function as the one underlined and capitalised in the key sentence. The fourth subtest is Vocabulary Learning (Szótanulás), which measures rote learning ability. In this part the students can study a list of 24 Swahili-Hungarian words for five minutes, then are given 20 Swahili words, and have to choose the correct Hungarian equivalent of these words from five alternatives.
The administration of the test takes 60 minutes. To increase the reliability of the test the instructions for the learners are recorded on a CD. The examiner’s task is to distribute the MENYÉT booklets, start the CD player, and make sure that the testees follow the instructions.
Research framework and objectives
155
The present study follows the traditions of the qualitative paradigm: instead of utilising statistical procedures it provides „thick description” of the data (Mackey-Gass 2005).
Like other qualitative experiments it does not justify a hypothesis, but sheds light on a particular phenomenon of foreign language learning without the intention of generalizing the findings.
The present article is based on two case studies (Hild 2007; Hild in press) where I assessed the response validity of the Language Analysis component of MENYÉT with think-aloud protocol. In this subtest the testee has to choose the correct translation of Hungarian sentences in an artificial language from four alternatives with the help of a group of sample sentences and words written in this language and their Hungarian equivalents. My aim was to find out what reasoning the four participants engaged in while responding to the items, and whether they acted upon according to the expectations and tried to find the right answer by extrapolating the necessary linguistic rules from the sample sentences. In my present study I compare the performance of the four participants of the two case studies, and examine what schooling- and age-related differences can be observed in the way they carried out the task.
Participants
In a study using think-aloud protocol the selection of participants requires extra caution on the part of the researcher. The samples need to be chosen purposefully (Li 2004).
Ideally, they should volunteer to take part in the experiment, and their anonymity must be guaranteed. In order to reduce the effect of the observer’s paradox and increase the trustworthiness of the study before the experiment the researcher should get to know the participants. Heeding these words of advice I offered four of my private EFL students the opportunity to take part in the study. They all gave a positive answer. I also informed them that their anonymity would be assured. As I have known them for more than a year, I could also minimize the distortion caused by my presence during the research. The reason why I chose these particular students is that their educational background and age were suitable for the purpose of the study.
One of the participants is a 32-year-old dentist, Kata. She has been my EFL private student for 18 months. At university she learned English for four years. At secondary school she attended a special German as a foreign language class. Towards the end of secondary school she managed to pass an intermediate-level language exam in German.
She also learned Russian for five years at school. In her self-report she states that she was good at all the school subjects, even at the ones she did not like, which were chemistry and geography.
The other participant is a 35-year-old carpenter. András has been running his own business making and selling wardrobes for ten years. He has been attending my English classes for one year. At primary school he learned Russian for eight years. In the first four years he also learned German, but he gave it up when he had the chance. At the vocational secondary school he did not have any foreign
156
language lessons. At school he was good at drawing and physics, but he did not like history and literature, because he had to mug too many dates and names, and these lessons were not „practical enough”.
János, the third participant, is a 56-year-old mathematician. He gave up his profession ten years ago and started his own family business, which he has been running successfully ever since. He has been attending my private EFL lessons for 4 years. At school he learned Russian for eleven years and Latin for four years. His favourite subjects were maths and physics, but he was not good at history, literature and chemistry.
The youngest participant, Máté, is a 17-year-old student of a secondary school with a good reputation. He has been learning English for three years. He has passed an intermediate-level language exam in German, which he started to learn when he was seven. In his self-report he stated to be good at maths, computer science and English, and weak at literature and biology. Since the experiment he has applied for and been admitted to the University of Economics.
Instruments
All the participants were given a MENYÉT booklet that contained all the four subtests and a separate answer sheet. The instructions of the test were provided on a CD player.
Their thoughts were elicited with think-aloud protocol (Gass-Mackey 2000), and recorded on a digital dictaphone. Think-aloud protocol is an introspective means typically used for investigating cognitive processes, during which the participants are asked to do a task and simultaneously verbalize their thoughts, i.e. to say aloud what they are thinking of. After the test I also gave them a data sheet which contained open questions on their age, qualification, profession, foreign language knowledge, and the school subjects they were good and bad at.
Procedures
In a qualitative study where the method of data collection is think-aloud protocol it is advisable to choose a familiar research environment where the participants do not feel insecure and vulnerable (Li 2004). Therefore, the experiments took place where we usually have our classes, namely in my home or in the students’ home. András and Máté seemed relaxed and comfortable before the test. However, Kata and János were a little bit more nervous. In Kata’s case this can be attributed to her usual test anxiety she experienced several times while attending the Medical University. János’s questions and comments before taking the test implied that he was more concerned about his performance, which might be imputed to the fact that he considered the test as a question of prestige.
As the pilot study carried out before the two case studies justified that there was no need to practice thinking aloud and completing the task concurrently I only explained
157
the participants, upon request and right before the test that they were supposed to say constantly what they were thinking of while taking MENYÉT.
Results and Discussion
Before taking the Language Analysis component, the test takers were given two minutes to familiarize themselves with the task by looking at a sample item. Kata, Máté and János understood the instructions and were able to find the correct solution of the sample item. However, as András did not think aloud, I had to remind him and explain to him that he needed to report on everything what was on his mind, which took a certain amount of the time that was given for the sample item. The instruction that he had to translate the Hungarian sentence into the new language in his head also confused him. He did not understand how to accomplish thinking aloud and translating in his head simultaneously. (In the extracts the English translations of the Hungarian comments that the participants made during the test are printed in uppercase):
Extract 1
ANDRÁS: „IT SAYS HERE THAT IT HAS TO BE DONE IN YOUR HEAD. SO DO I HAVE TO DO IT ALOUD IN MY HEAD?”
His hesitation contributed to the fact that he could not come up with an answer within the allocated time, which he found amusing.
When the voice on the CD player asked the participants to begin doing the test Kata, Máté and János had no problem with following the instructions. They all started off by reading the sample sentences and words at the top of the page. Kata spent the most time studying them. At this point she even tried to extrapolate some of the linguistic patterns of this artificial language. However, András seemed completely lost, was not sure which were the sample sentences, and where to write his answers; and as before he turned to me for help. When I informed him again that I could not be of any help he laughed and stayed quiet for a few minutes, so I asked him anew to think aloud. This time he complained that it was not very convenient for him to report on his thoughts while doing the exercise, because it took up too much of his time.
In the following part of the experiment all the participants tried to find the correct answers by extrapolating the necessary morphological, syntactic and semantic rules of this artificial language from the sample sentences and words. When they experienced difficulties they often applied the exclusion of the incorrect options with the help of the previously extrapolated rules.
Out of the four participants András was the slowest. After getting used to the idea of thinking aloud he could attend to twelve out of 20 items within the allocated 15 minutes, out of which he managed to solve ten correctly. He could infer the more complex linguistic patterns as well (e.g. negative and merging of words and omission of letters). The two mistakes he made were due to the fact that he did not notice that the sentences were in the past tense, so he incorrectly picked the options in the present.
158
However, later on he managed to find out how to form the past tense, and he even realized that he had earlier ignored this rule, but did not take the trouble to go back and correct his mistake. When András ran out of time he did not mark randomly answers on the separate answer sheet, even though the instructions on the CD player reminds the test-takers about the possibility of using this test-taking strategy.
Kata was the quickest, and could finish this subtest in eight minutes. Consequently, she had plenty of time to check all her responses, but found no mistakes, because there were none. As she herself stated, she applied exclusion not only when she faced difficulties, but wherever she could to find the answer faster:
Extract 2:
KATA: “WHO DOES IT? WHEN DOES HE DO IT? IS IT IN THE NEGATIVE?
AND YOU CAN EXCLUDE A WHOLE LOT OF ALTERNATIVES, AND YOU CAN DECIDE MUCH FASTER WHAT IT STARTS WITH, AND IN THIS WAY, HALF, OR ALMOST HALF OF THE CASES DROP OUT.”
Owning to the rules she had extrapolated from the sample item and while studying the sample sentences, in the beginning Kata’s information-processing was so rapid she did not have time to report on her thoughts.
Máté was the second quickest among the participants, and could finish off in nine and a half minutes. In the remaining time he checked his responses, and found three mistakes. In two out of three the problem was that, similarly to András and János, he did not realize that the sentences were in the past, and therefore picked the present form. In the third item he used the verb watch instead of chase. However, he failed to notice one mistake. In this item after picking an answer by excluding options he did not consult the sample sentences again, but incorrectly followed the Hungarian rules of the possessive case, where, contrary to this artificial language, the possession comes first with the possessive suffix at the end. (In the following extract the English translation of the Hungarian comments that the participant made during the test are printed in uppercase, the words of the artificial language are indicated by lowercase bold letters, whereas the Hungarian words that the participant read from the test sheet are printed in uppercase bold letters):
Extract 3
MÁTÉ: „YOUR CAT, SO IT NEEDS TO CONTAIN xa, MOUSE, WHICH IS ciud, AND CHASE, WHICH IS bo. IT IS DIFFICULT TO DECIDE WHICH IS CORRECT. IT MIGHT BE ’C’, meuxa ciud bo.”
In another item after managing to choose the only possible correct solution with the help of exclusion, he even managed to establish the new rule, which he could then apply accurately in the following sentences. Similarly to Kata, in the beginning his information-processing was too fast so that he did not have time to think aloud.
159
János came in third in the group. He could attend to 15 out of 20 items. He made four mistakes. In one of the items he chose the correct option, but marked a wrong one on the separate answer sheet. Once, similarly to András and Máté, he did not realize that the sentence was in the past, and wrongly used the present tense instead. In the third case, like Máté, he confused the verb watch with chase. In the fourth case he failed to notice an unusual morphological rule of this artificial language. Throughout the Language Analysis subtest János was very meticulous. In five out of 15 items he attended to after picking an option he immediately checked his train of thoughts and solution once again. Similarly to András, in the end he did not seize the opportunity to increase his test scores, i.e. he did not mark randomly responses on the separate answer sheet.
As for the protocol of thinking aloud, with the exception of András, three participants seemed to understand and be comfortable with the idea. During the experiment I hardly ever needed to remind Kata and János to report their thoughts while taking the test. Máté was more inclined to forget about this additional task and remain silent while thinking, but following the reminder he was ready to conform to the instruction. However, András did not really understand the idea of thinking aloud.
The first few minutes were spent explaining to him how to think aloud even if it was inconvenient and slowed him down. But following this initial hesitancy he also managed to verbalize his thoughts.
Conclusion
In the Language Analysis component of MENYÉT irrespective of their age and educational background all participants took the trouble and tried to find the correct answers by extrapolating the necessary linguistic rules from the sample sentences and
In the Language Analysis component of MENYÉT irrespective of their age and educational background all participants took the trouble and tried to find the correct answers by extrapolating the necessary linguistic rules from the sample sentences and