The reliability of RCA at Stage 2. Perhaps the most exciting part of this stage of our investigation of reference in RAs is checking the reliability of the analysis

139

Y Z

57 3189 native English-speaking children

58 the children

60 the low working memory group

68 the average working memory

group

Figure 6. A splitting reference chain

The opposite of splitting is merging. This happens when one referring item points to two different presupposed items. This is shown in Figure 8 (from the analysis of RA9), where both text types refer to the simplified text types and the authentic text types as well.

F G

12 simplified texts authentic texts

13 the simplified text types the authentic text types

16 both text types

17 the texts

Figure 7. A merging reference chain

After the merge in line 16, the chain continues as a linear chain, which splits into two again later in RA9. Merges and splits will therefore be summarized in the results as phenomena (the number of times they happen), and not as different types of chains.

Those in Figure 6 and 7 would each count as one occurrence.

The analytical procedure, as explained in Chapter 4 (Section 4.3), also takes into consideration the lexical categories of nominals (see Table 16 for the lexical aspect of cohesive reference). The summary of the analytical procedure is also in Appendix H.

140

The reason why it is so interesting is that comparison of the results of reference in longer texts has been nearly impossible up to this point. As we have pointed out in Chapter 5 of this paper, Halliday and Hasan’s (1976) analysis was not possible to work with when it came to longer texts, comparing several analyses was even more complicated. In the present shape of the analytical tool for Referential Cohesion Analysis, chains of reference were represented by columns in an Excel table. This representation made the comparison of several analyzes of the same text easily comparable. A fellow researcher from the PhD program, Ildikó Szendrői, was kind enough to devote a great amount of time to analyzing RA3 in my RA corpus, which made it possible to check inter-coder reliability. The text of RA3, is 153 sentences long;

one of us identified 205, the other 236 cohesive ties organized in altogether 39 reference chains of which 26 were exactly the same in our analyses. In the remaining 13 chains there were a number of differences. The two analyses were copied into one table with identical or similar chains in parallel, thus we could count the ratio of identical ties, the result of which is an 85.2% match. This result is much more convincing than what we had at Stage 1 with the abstracts, where only 54% of the ties were identified by both raters and only 18% of the ties were analyzed consistently in the same way.

Differences might be due to that fact that we are both humans, we have different interpretations of ambiguous items and different pieces of information will be important for us from texts even in different circumstances or periods of our lives. However, this factor should not account for more than 2-3% of the deviations in the analysis; hence, we looked more closely at the chains of reference. What we found was a partial reappearance of some of the problems we have discussed earlier, but with fewer instances here. Specifically, the case of ellipted phrases and the interpretation of lexical relations are described here.

141

Ellipsis in a presupposed item in the same sentence: in one analysis the referring items their and they had the full referent 839 male and female university students as their presupposed item, but in the other, it was understood as pointing to the ellipted nouns in the same sentence and were not listed in the RCA table. (In the examples:

s=sentence, underlined=presupposed item, boldface=referring item from the particular sentence)

s. 4. There were 839 male and female university students enrolled in an introductory psychology class who had access to review class lectures via the VLH.

s. 6. Approximately 20% (ellipted: of the students) used the resource, and 18%

(ellipted: of the students) completed a five-item survey tapping their perceptions of whether the VLH enhanced learning or increased grades; and whether they wanted the resource in other courses

Lexical relations were again problematic, especially when it came to part-whole relationships. In the list of examples below, the referring items in one analysis were considered cohesive, and in the other, they were not:

students ← the student number the course ← the course conclusion slides ← the slide titles

the lecture ← the lecture content this university ← the faculty

Besides the above differences, items with determiners were occasionally missing from one analysis: most enrolled students, each student. A new problem that arose here was the status of proper names with a definite article: one analysis identified a proper name with a definite article as a cohesive tie with its previous mention, the other did not. Another issue is whether or not items such as the present study refer to the manuscript in the same way as this paper or this study.

142

Finally, two splitting chains were not identified as such in one analysis;

nevertheless, the cohesive ties all appeared, but under the same presupposed item.

Figure 8 shows one of these instances, with the two different chains from the two analyses in parallel:

Chain 1 Chain 2

the Winter 2003 semester the Winter 2003 semester

the semester the semester

the first midterm the first midterm the first midterm the first midterm the second midterm,

the end of semester

the second midterm

Figure 8. Sample from the reliability analysis at Stage 2.

Occasional differences occurred with items like the computer which was ambiguous between exophoric and cohesive.