The structure of experiments in cognitive linguistics

A very important insight of the current literature on scientific experiments is that neither single experiments nor repetitions of the experimentation process are capable of yielding ultimate and unquestionable results. It is not only the previous considerations and the planning of the exper-iment which are fallible – the control of the experexper-imental process and the evaluation of the results are to some extent unavoidably uncertain as well. Therefore, experiments are open pro-cesses in the sense that, in possession of new pieces of information, they may be continued, modified, or even discarded. As we have seen in Section 3.1, according to recent trends in the philosophy of science, the reliability of the outcome of an experiment depends on the reliability of its components, as well as the fit between them and pre-existing knowledge. To find this fit, one has, in most cases, to turn back to earlier stages of the experimentation process and modify some component. Every component can be revised and the revisions have to be repeated again and again until there is mutual support among the constituents. See Figure 2.

experimental design; Figure 2. The basic structure of experiments

As we have already mentioned in Section 3.1, the idea of achieving a mutual support among the components of the experiment as an arbiter of the acceptability of the experimental results involves the risk of circularity and may lead to the experimenter’s circle. This problem, as Kertész & Rákosi (2009) has shown, can only be solved heuristically. This means that if a process returns to the start in such a way that it leaves it unchanged by failing to re-evaluate the information content at one’s disposal, then it is ineffective and does not bring one closer to the solution of the problems raised. As opposed to this, with cyclic processes, “one indeed returns to ‘the same point’ but does so at a different cognitive level” (Rescher 1976: 119), since a modified, prismatically re-evaluated, qualitatively new information state is created (see also points (ix)–(xi) in Section 4.1 in Kertész & Rákosi 2009, Section 10.4 in Kertész & Rákosi 2012, as well as Rescher 1987). Accordingly, cyclic argumentation is effective. From this it follows that if the evaluation of the results and components of the experimentation process systematically ignores potentially relevant data, does not examine alternative interpretations, does not check possible error sources, or leaves unclarified relevant factors which could de-crease the reliability of the results, then we are faced with the experimenter’s circle. If, how-ever, the process is prismatic in the sense that one continuously changes the perspective from which the pieces of information constituting the context are evaluated (cf. Rescher 1987, 1977;

Section 10.5 in Kertész & Rákosi 2012), and alters our informational state by extending it with new pieces of information or by revising it, then it is the experimenter’s cycle – an effective and fruitful enterprise of gaining new information about the world.³⁶

Nevertheless, this mutual support does not guarantee the certainty of the results; rather, it is a sign of their plausibility. It is reasonable to accept them on the basis of the available infor-mation but one should never forget that there may always be systematic errors, other alternative explanations etc. that have not been taken into account. To reduce the amount and impact of the latter and to increase the plausibility of the results of the experiment, one has to actively seek for possible weak points – that is, one has to reflect on every detail of the experimentation process from its planning to its evaluation with the help of the strategies proposed by Franklin and by elaborating further ones.

36 For similar views, cf. Nickles (1989), Pickering (1989), and especially in linguistics, Pullum (2007).

In this vein, let us take a closer look at the process of searching for the best fit among the components of the experimental process from the point of view of the experimenter.

Hypotheses used in the experimental design, the theoretical model of the phenomena and of the apparatus make up the starting point of the experimental process. They are not true with certainty but they are supported to some extent by theoretical considerations, by earlier exper-iments, or are simply (reasonable) conjectures. They allow for a rough estimation of the out-come of the experiment. After the experimental procedure, in possession of the perceptual data, this preliminary guess may be strengthened. Nevertheless, it may happen that the perceptual data cannot be interpreted properly, or they seem to be in conflict with the predictions. In such cases, the reliability of the previously accepted hypotheses also has to be revised.

The interpretation and authentication of the perceptual data may also indicate shortcom-ings in the experimental procedure, in the experimental design, or in the theoretical model of the phenomena or of the apparatus. Therefore, all facets of the experiment conducted have to be re-examined, and, if it seems to be necessary, control experiments have to be carried out, or the experimental design has to be modified and the experiment repeated. Moreover, even the interpretation or the authentication of the perceptual data itself may be faulty and be in need of modification.

From this it follows that revealing the connections between the statements capturing dif-ferent aspects of the experimental procedure and their analysis, as well as the comprehensive-ness of the checks and cross-checks are of crucial importance.

This characterisation of the experimental process will motivate us to raise the hypothesis that experiments are cyclic processes organised and conducted by an argumentation process which tries to clarify the relationship among hypotheses of the experimental design, the theo-retical model of phenomena, the theotheo-retical model of the experimental apparatus, the theory under test and its rivals, as well as statements describing the events of the experimental proce-dure, or which capture the results of the interpretation and authentication of perceptual data etc. This motivates us to modify Figure 2 as shown by Figure 3.

experimental design;

theoretical model of the phenomena;

theoretical model of the apparatus

experimental

proce-dure perceptual data

revision plausible

argumen-tation confrontation of the

ex-perimental data with the theory/rival theories

experimental data

authentication and interpretation of

perceptual data

transformation experimental report

Figure 3. The structure of experiments – extended³⁷

The argumentation process organising the conduct and control of the experiment is a central issue in judging the reliability of the experimental results. This argumentation process does not consist of deductive inferences because it takes into consideration the uncertainty of the state-ments as well. It is not public; rather, it is a private affair of the experimenters. Despite this, it is indirectly influenced by the public norms applicable to experiments. Thus, experimenters have first to convince themselves of the reliability and acceptability of the outcome of the ex-periment and, after this, they have to persuade the scientific community as well. This means that the researcher has to transform this private argumentation process into a public experi-mental report.

If we turn to the reader’s perspective, we can establish that the evaluation of experimental results can only start from the experimental report, which is an edited, transformed version of the non-public argumentation process. While the latter is part of an ontologically complex pro-cess of scientific experiment, the former is purely argumentative. It is a mixture: it contains elements or traces of the original argumentation process as well as the argumentative tools needed to make this reduced set of information coherent, comprehensible and persuasive for the reader.

In the next section, we will take a closer look at the peculiarities and function of argumen-tation in experiments in cognitive linguistics and present tools which allow us to reconstruct it and evaluate its acceptability.

37 Simple arrows indicate successive stages of the experimental process; dotted arrows signify the non-public argumentation process which organises the experimental process.