LEXICAL RELATIONS
6.5 Methods of analysis
The main purposes of this study were the following:
to check the validity and reliability of the tool for Referential Cohesion Analysis through inter-rater analysis;
to explore what data can be obtained about referential cohesion in research articles,
to see how this data is similar to or different from the results that can be obtained about referential cohesive ties using the original taxonomy (Halliday &
Hasan, 1976),
to describe analytical problems and decisions;
and finally, to collect those hypotheses and further research questions that occurred during the analysis.
This part of the research was quantitative in the sense that the data obtained about the cohesive patterns in the analyzed texts were quantified. Presenting the results of a cohesion analysis in this way was useful to allow several texts from various genres to be compared. Nevertheless, qualitative interpretation of data obtained from the analysis of the ten research articles was inevitable. In its procedures, the study relied as much on a bottom-up approach to text analysis (working from corpus data) as on top-down approaches (in observing research articles as a whole, looking for characteristics within the units into which they are divided, as well as seeking emergent patterns in the
134
chains of reference). This being a pilot study of the proposed analytical tool, it we expected that probably more questions would be raised during its first implementation than can be answered at this stage. Indeed, these questions were seen by the author as highly relevant findings in guiding the gradual process of the development of the tool.
The two main research questions that guided the investigation in this exploratory study can be formulated as follows:
1. In what ways is the modified application of Halliday and Hasan’s (1976) taxonomy for a corpus of 10 research articles similar to or different from the originally proposed application (Halliday & Hasan, 1976)? More specifically: How does this modification of the analytical procedures affect the construct of referential cohesion?
2. What genre characteristics of research articles can be identified on the basis of the proposed (quantitative and qualitative) analysis of this corpus as regards the types of referring items, the cohesive ties they typically establish, and the patterns and distribution of cohesive chains of reference?
While we did expect to obtain results concerning the characteristics of RAs (Question 2), this Chapter focuses on the results that relate directly to the analytical procedure itself. Findings concerning the description of RAs as a genre will be described in Chapter 7.
The discussion of the theoretical background to the specific categories of items that we are focusing on here is contained in Chapter 3, Table 8. The main types of cohesive reference are repeated here for ease of reference:
135
1. Personal reference: a) 3rd person pronouns (e.g. she, him, theirs); b) possessive determiners (e.g. his, her + NP)
2. Demonstrative reference: a) determiners of place and time (here, there, then); b) central determiners (e.g. the, this, those + NP)
3. Comparative reference: a) pre- and postdeterminers (quantifiers: all, both, many, several, either, etc. + NP; semi-determiners: same, certain, such, etc. + NP); b) adjectives and adverbs of comparison (e.g. different(ly), so + NP)
Again, the analysis starts with numbering the sentences, including the title. After identifying the sentence boundaries, the procedure slightly deviates from Halliday and Hasan’s (1976): instead of looking only for elements that are cohesive, all the items that may have the property of reference are highlighted by a simple computer search and replace function (the items from Table 8): this way, no referring items are overlooked.
Comparatives or other phrases, which contain more than one such item (e.g. the same, this difference), were counted as one item and was categorized according to the first item appearing in the sequence.
The next step in the analysis is the first reading of the text – here, the research article – to identify and cross out all those referring items that are used non-cohesively.
These are mainly of two types; first, items that have their referent within the same sentence are not cohesive as they do not contribute to the structure of the text, but link the parts of their sentence or clause, the constituents of which ‘obviously “cohere” with each other, by virtue of the structure’ (Halliday & Hasan, 1976, p. 6). Second, there are exophoric items, which point to a referent not found in the text. Reference is established by those referring items that can only be interpreted by using information from outside the sentence. Only those referring items will be regarded cohesive that refer to a presupposed element, or referent outside their containing sentence (which may be of any length from a word to various sentences).
136
After eliminating non-cohesive items, a new Excel worksheet was created for each of the analyzed RAs. In the worksheet, each row represents one sentence in the text. Thus a text of 256 sentences occupies exactly 256 rows, the title being the first. The analysis proceeds sentence-by-sentence. Each column in the worksheet (A, B, C ... AA, AB, AC ... etc.) represents one cohesive chain, that is, a referent (blue in the table) and all the referring items that point to it. A sample analysis of sentences 85 to 91 from RA9 is shown in Figure 5 with the relevant text extract. The words in bold are the ones that need a referent somewhere in the text, but if their reference is specified within the same sentence or is exophoric, they are crossed out. The remaining referring items appear in the chart under their relevant presupposed item. These presupposed items are underlined in the RA extract. The referring items in this text are the pronouns it and its, the definite article, and the demonstrative these.
The two chains of cohesive ties in Figure 5 are linear. The first chain (AG) contains four cohesive ties, and the first tie of the second chain (AH) appears in this table. An advantage of this representation of cohesive items is that it can capture the concept put forth by Halliday and Hasan (1976, p. 332) that it “does not much matter for cohesive purposes (however interesting it might turn out to be in other respects) whether she occurs one or half a dozen times within the sentence”, because in the cohesion analysis table, one slot can be filled with “half a dozen” items referring to the same entity. Still, it will appear as one cohesive tie in a chain and we still do not lose (leave out) any items in the analysis. (With some imagination, a slot could be extended into a third dimension in the table, so that synonyms and pronouns appear behind the reference item – which would result in a table too complex to be meaningful. Still, the idea might help to give an understanding of how we distill the whole text into a simple reference-structure skeleton.)
137 Cohesive chains
AG
AH Presupposed
items Sentences
from the research article
85 Coh-Metrix
86 its
87
88 the system
89 it
Referring items
90 it
several metrics of word and language
characteristics such as word frequency, parts of speech, concreteness, polysemy, density of noun phrases, and familiarity measures
The relationship between these measures in AH91 and the sentence fragment in AH90 is a cohesive tie
91 these measures
85. Coh-Metrix is a computational tool that measures cohesion and text difficulty at various levels of language, discourse, and conceptual analysis.
86. The goal of its designers was to improve reading comprehension in classrooms by providing a means to write better textbooks and to match textbooks to the intended students more appropriately (Graesser et al., 2004; Louwerse, 2004; McNamara et al., 2002).
87. Coh-Metrix is an improvement over conventional readability measures because it provides a detailed analysis of language and cohesion features and eventually matches this textual information to the background knowledge of the reader (McNamara et al., 2002).
88. The system integrates lexicons, pattern classifiers, part-of-speech taggers, syntactic parsers, shallow semantic interpreters, and other components that have been developed in the field of computational linguistics (Jurafsky & Martin, 2000).
89. It analyzes text cohesion in several ways, including coreferential cohesion, causal cohesion, density of connectives, latent semantic analysis metrics, and syntactic complexity.
90. For the purposes of comparison, it also includes standard readability measures such as Flesch-Kincaid Grade Level and several metrics of word and language characteristics such as word frequency, parts of speech, concreteness, polysemy, density of noun phrases, and familiarity measures (Graesser et al., 2004).
91. Many of these measures parallel the linguistic features used to support arguments for both sides in the debate over using authentic or simplified texts for L2 reading.
(RA9)
Figure 5. Referential Cohesion Analysis: Sample analysis 1.
138
A weakness of Halliday and Hasan’s (1976) original taxonomy is that it assumes that chains of reference always display linearity in the cohesive chains of reference, which seems to be insufficient to capture the complexity of referential relationships.
This is also recognized in Hasan’s (1984) analysis of cohesive harmony, where – as we mentioned earlier – she describes other patterns, such as the disjunctions and conjunctions of chains. One of the most relevant advantages of our proposed analysis is that it can account for patterns of reference that have posed serious problems and sources of inconsistencies in the analysis for the researcher intending to apply Halliday and Hasan’s (1976) taxonomy. The patterns of cohesive relationships of more than two ties (that is, chains) in the corpus can be described by three main patterns: linear, splitting, merging. While more complex patterns can be observed, they are basically built up of the same basic patterns, and are relatively rare in this corpus.
A linear chain can be described as one in which the presupposed item and all the items that refer to it occupy one single column. This happens when the meaning of the referring item covers the same entity or phenomenon (has the same real-world referent) as the presupposed item. In Figure 5 there are two linear chains, AG and AH. Figure 6 shows a splitting chain. In a splitting chain, two different items together cover the reference of the same presupposed item. In other words, the meaning of each referring item in such a chain only refer partially to the presupposed item, but their referents are not the same. Figure 6 shows two columns from the analysis of RA4. In line 57 the presupposed item 3189 native English-speaking children refers to the whole group of participants in the research discussed in the article. In the next row, the children refers to this same group, however, in rows 60 and 68 the chain splits into two subsections to differentiate between two parts of the same group. Therefore, the referent is depicted in Figure 6 in the first row as occupying two columns.
139
Y Z
57 3189 native English-speaking children
58 the children
60 the low working memory group
68 the average working memory
group
Figure 6. A splitting reference chain
The opposite of splitting is merging. This happens when one referring item points to two different presupposed items. This is shown in Figure 8 (from the analysis of RA9), where both text types refer to the simplified text types and the authentic text types as well.
F G
12 simplified texts authentic texts
13 the simplified text types the authentic text types
16 both text types
17 the texts
Figure 7. A merging reference chain
After the merge in line 16, the chain continues as a linear chain, which splits into two again later in RA9. Merges and splits will therefore be summarized in the results as phenomena (the number of times they happen), and not as different types of chains.
Those in Figure 6 and 7 would each count as one occurrence.
The analytical procedure, as explained in Chapter 4 (Section 4.3), also takes into consideration the lexical categories of nominals (see Table 16 for the lexical aspect of cohesive reference). The summary of the analytical procedure is also in Appendix H.