Facilitating function-based methodologies in ELF research: the reliability of pragmatically annotated VOICE data
Research on intercultural communication through English as a lingua franca (ELF) tends to employ form-to-function methodologies for corpus-based pragmatic analyses. These take specific linguistic forms as starting points to examine the variety of communicative functions for which they are used (Rühlemann & Aijmer, 2015, p. 9). The reverse methodological proceeding from function to form is less influential. Function-based research is ideal for studying variable and adaptive form-function relations as characteristic of intercultural ELF settings but fundamentally requires pragmatic annotation. This annotation is resource-intensive and, as Artstein and Poesio (2008, p. 555) report, often perceived to be less reliable than other forms of corpus annotation. The skepticism over the reliability of pragmatic annotation, however, coincides with scholarly acknowledgement of the desirability of pragmatically annotated data for corpus-pragmatic research (cf. e.g. Clancy & O’Keeffe, 2015, p. 251).
To further facilitate the desired function-based analyses in ELF research, a scheme to annotate pragmatic functions in data from the Vienna-Oxford International Corpus of English (VOICE) (VOICE, 2021) has been designed. The aim of this paper is to report on the reliability of the annotated data. The paper first describes the annotation system and provides passages from VOICE transcripts annotated for pragmatic functions. It then traces cases of inter-annotator agreement and disagreement. Finally, Cohen’s Kappa and Krippendorff’s Alpha coefficients are presented as statistical measurements indicating the reliability of the functionally annotated ELF data against the backdrop of which function-based studies of ELF interactions for intercultural communication research may be conducted.
References
Artstein, R., & Poesio, M. (2008). Inter-coder agreement for computational linguistics. Computational Linguistics, 34(4), 555–596. https://doi.org/10.1162/coli.07-034-R2
Clancy, B., & O’Keeffe, A. (2015). Pragmatics. In D. Biber & R. Reppen (Eds.), The Cambridge handbook of English corpus linguistics (pp. 235–251). Cambridge University Press.
Rühlemann, C., & Aijmer, K. (2015). Introduction: Corpus pragmatics. Laying the foundations. In K. Aijmer & C. Rühlemann (Eds.), Corpus Pragmatics (pp. 1–26). Cambridge University Press.
VOICE. (2021). The Vienna-Oxford international corpus of English (version 3.0 online). https://voice3.acdh.oeaw.ac.at