The effect of the wiki project on the participants’ language proficiency

The main purpose of using the wiki was to engage the students more intensively, as well as to enhance self-study and personal learning in order to help students prepare for extended language learning beyond the obligatory three terms of language studies at the college. However, as deeper engagement with the task can lead to increased proficiency (Golonka et al., 2014); the use of the wiki was expected to foster language development as well. In this section first students’ language proficiency in English will be described at the beginning of the course based on the results of the Needs analysis questionnaire, the Placement test and the Self-assessment grid for the Common Reference Levels in the Common European Framework of Reference for Languages (2001). Subsequently, students’

perceived language development will be discussed including the comparison of the findings of the two questionnaires of assessment of language development and the Self-assessment of language proficiency. Finally, the results of the Language proficiency test will be presented to characterize students’ language proficiency at the end of the course.

As for students’ language knowledge at the time of starting the course, they had all passed a B2 level language exam and four students even had a C1 level exam in general English (Nóra, Enikő, Dorina and Stefi), which means that they had already fulfilled the language requirement in English for obtaining a degree. In the Needs analysis questionnaire 17 students rated their skills on a five-point scale between 3.23 (listening) and 4 (reading) on the average and indicated that they would like to improve all skills during the course, reading the least and grammar and speaking the most (See Table 43). Their aims for studying English at the college included passing the B2 professional or C1 exam (13 students), learning to

182

speak fluently (4 students), preparing for future work in general (2 students) or abroad (9 students). Most participants indicated that they watched films with subtitles to improve their English (13 students), listened to music or read books (3 students each).

Table 43

Mean scores for the results of the needs analysis questionnaire (N=17)

reading speaking pronunciation writing grammar listening Perceived rate of

skills 4 3.5 3.37 3.36 3.26 3.23

Perceived need for

improvement 3.36 4.12 4.1 4 4.12 3.97

Before the course started 17 students wrote the placement test with the average result of 78.5%, ranging from 75% to 80%, which was higher than the average of all students studying English (64%). Besides the Placement test and the Needs analysis questionnaire, 17 students also assessed their language proficiency with the help of the self-assessment grid for the Common Reference Levels in the Common European Framework of Reference for Languages (2001) at the beginning of the course. The proficiency levels were first converted into numbers (A2=1, B1=2, B2=3, C1=4, C2=5), then the mean scores were calculated for each skill (See Table 44). The results show that students rated their writing, reading and listening skills between level B2 and C1, while their speaking skills between B1 and B2. To compare data from the beginning and the end of the course, mean scores for the 11 participants who filled in the grid December 2013 were also calculated for September 2012.

For most skills the mean scores of the ratings were the same (listening) or higher (reading, spoken interaction, spoken production) in 2013. The only exception is writing, which students assessed slightly lower at the end of the course than at the beginning. Although no statistical analysis was carried out because of the small sample size, a comparison of individual students’ scores indicates that six students rated all their skills at the same or a higher level, four students at the same level on average but with different points for the skills and only one student (Tibor) rated two skills (spoken production and writing) one level lower after the

183

course. As for writing, all the eleven participants rated it as level B2 in 2012. However, in 2013 only four students rated it as B2, while two students (Inez and Stefi) perceived a development to C1, five students indicated a negative change to B1 (Odett, Tibor, Livi, Dorina and Zsófi).

Table 44

Mean scores of the results of the Self-assessment of language proficiency questionnaires listening reading

spoken interaction

spoken

production writing September 2012

(N=17) 3.5 3.25 2.75 2.69 3.12

September 2012

(N=11) 3.36 3 2.55 2.45 3

December 2013

(N=11) 3.36 3.18 3.27 3 2.72

Notes: A2=1, B1=2, B2=3, C1=4, C2=5

A possible reason for the lower rating may have been caused by the introduction of formal writing during the course, which included formal letters, emails, memos, reports and leaflets. While students were probably assessing their writing skills in general English at the beginning, writing skills were possibly perceived as skills for professional writing at the end of their studies.

Besides their level of proficiency, students also assessed their perceived language development on a five-point scale after the second (N=15) and the third term (N=12). The comparison of the results of the twelve participants who filled in both questionnaires shows a higher perceived development in the third term than in the second (Table 45). While in the first survey five students chose 4 or 5, i.e. their perceived language development was quite a lot or a lot (M=3.42), in the second one it was higher with eight students (M=3.75). The results were very similar for all the subskills and areas, apart from the perceived development of their professional vocabulary, where the mean score was higher than 4 in both terms (M=4.33 in May and M=4.25 in December). This could be caused by the fact that they had never studied that area before, consequently progress could be perceived more easily. All

184

students reported an improvement of all skills in each term, with the same or a slightly higher rate of development in the third term.

Table 45

Mean scores of the results of the Self-assessment of language development questionnaires listening reading speaking writing grammar

professional vocabulary

English knowledge May 2013

(N=15) 2.73 3.4 3.13 2.87 3.27 4.2 3.33

May 2013

(N=12) 2.75 3.5 3.17 2.83 3.33 4.33 3.42

December

2013 (N=12) 3.42 3.58 3.42 3.17 3.42 4.25 3.75

Finally, students’ language proficiency was measured at the end of the course by a language proficiency test, which was developed by four lecturers of the college. All students received a mark 5 for the three subtests and the results were above the mean score of all groups studying English (Groups A to S). Table 46 shows the mean scores for the placement test and the language proficiency test for all groups and for the top three groups of first year students in 2012, including two mean scores for Group C (the participants of the study), one for all students who have written the tests, the other for the eleven students who wrote both tests.

Table 46

Mean scores of the results of the placement test and the language proficiency test language proficiency test

placement

test listening

use of

English reading Groups A to S

N=228 64%

260.2 264.1 267.8 Group A, N=12 89.7% 269.4 302.9 313.5 Group B, N=17 82.4% 296.5 310.2 337.3 Group C, N=13 78.5% 288 310.8 358.7 Group C, N=11 75% 279.5 304.5 351

A comparison of the mean scores shows that although in Group A and B students had higher scores in the placement test than students in Group C, their results in the language proficiency test are only partly higher. In Group C scores were better than in Group A in all

185

three parts of the test, and higher than Group B in the reading part even for the final eleven participants of Group C.

To sum it up, students’ language proficiency was around level B2 at the beginning of their studies. They had all passed a B2 language exam before the course and rated their writing, reading and listening skills higher, while their speaking skills slightly lower. As for their language proficiency development during the course, they rated their skills the same or higher in 2013, with the exception of writing, which students assessed slightly lower at the end of the course than at the beginning possibly due to the introduction of formal writing during the course. Similarly, all students reported an improvement of all skills in the second and the third term, with the same or a slightly higher rate of development in the third term. At the end of their studies students performed very well in the language proficiency test with higher scores than the best group within their course. Thus, students’ language proficiency development has clearly been demonstrated by the results of Placement test, the Self-assessment of language proficiency questionnaire, the Self-Self-assessment of language development questionnaire and the Language proficiency test. However, as the wiki project constituted only one part of the course, which included several further tasks as well, this development cannot entirely be attributed to the use of technology.