AVERAGE COLLOCATION FREQUENCY AS AN INDICATOR OF SEMANTIC COMPLEXITY

Transcription

1 AVERAGE COLLOCATION FREQUENCY AS AN INDICATOR OF SEMANTIC COMPLEXITY Nikolaos K. Aagostou Departmet of Computer ad Iformatio Scieces Uiversity of Strathclyde Richmod Street, Glasgow, G1 1XH UK George R S Weir Departmet of Computer ad Iformatio Scieces Uiversity of Strathclyde Richmod Street, Glasgow, G1 1XH UK george.weir@cis.strath.ac.uk ABSTRACT The preset paper elaborates the ature of the Average Collocatio Frequecy ad proposes specific roles for this measure. I additio to its potetial as a factor i gaugig readability, we propose that the ACF affords a practical measure for estimatig the laguage complexity of texts for use i Eglish laguage teachig. Particularly, i the cotext of Eglish as a Secod Laguage, collocatioal cotet represets a dimesio of laguage sophisticatio that is especially challegig to learers. For this reaso, the ACF has potetial as a discrimiator of texts that may allow us to match learig materials to differig levels of learer. 1. INTRODUCTION I a previous paper (Aagostou ad Weir, 2007a), we itroduced the cocept of Average Collocatio Frequecy (ACF) ad argued for its use i estimatig the readability of texts. The ACF measure is built upo two elemets; corpus-derived frequecy lists ad the liguistic pheomeo of collocatios. To clarify this relatioship we eed to review the characteristics of existig readability formulae ad collocatios. 1.1 Readability formulae Readability formulae work by usig quatifiable textual aspects, i order to estimate the difficulty iheret i that text. Covetioally, these tools focus o word legth ad setece legth, or variatios o these costructs. Word legth fuctios as a idicator of sematic difficulty, followig Zipf s Law (Zipf, 1949) which idicates that the most frequet words i a laguage, ad hece the most familiar, ted to become shorter. I cotrast, setece legth fuctios as a idicator of sytactic difficulty, sice loger seteces impose heavier demads o memory ad metal processig o the reader (Bormuth, 1966, cited by DuBay, 2004). These aspects are well fouded i readability studies (Dave ad Chall, 1995), but suffer the limitatios faced by early readability researchers. Most readability formulae were developed i the period from the 1940s to the 1970s, with a orietatio towards maual use. Nowadays though, corpora cosistig of a trillio words (Fraz ad Brats, 2006), computers capable of utilisig such vast datasets ad a whole ecosystem of software tools for textual maipulatio are available, removig the limitatios that shaped the iitial stages of readability work. Such techologies make explorig the use of more geeral, laguagelevel variables possible. Such ewer factors may iclude word frequecy. The frequecy of words, as derived from large referece corpora, reflects a viable factor i estimatig readability sice more commo words are likely to be familiar to more readers. Thereby, a text composed maily of highly commo words is likely to prove more readable (more comprehesible). I spite of the plausibility of employig word frequecy i readability measures, there are curretly few examples (cf. Weir ad Ritchie, 2007; Steer et al., 1988). Our ow approach to readability estimatio takes a similar lie of reasoig ad assumes that icludig objective laguage-level criteria, such as frequecy lists extracted from corpora, ca improve the accuracy of readability formulae. The specific additioal factor that we recommed is based ot upo word frequecy, but o the frequecy of word collocatios.

2 1.2 Collocatios I atural laguage, words are ot combied radomly. Although costraied by grammar ad sytax, words also have prefereces. Firth (1957) amed such preferetial word combiatios collocatios. For istace, i Eglish it is proper to say strog tea but ot powerful tea. Oe ca say broad daylight but ot bright daylight. Some forms are covetioally acceptable ad others are ot, ad this appears to have arise arbitrarily i the course of laguage evolutio. There is little cosesus o what costitutes a collocatio ad most researchers adopt a defiitio that best fits their perspectives. I the cotext of our research, we regard word combiatios as collocatios o the basis of statistical associatio measures. Such measures determie collocatios by estimatig the stregth of bod betwee differet words. Some liguists argue that collocatios rather tha idividual words are the fudametal buildig blocks of a laguage. Regardless of whether this holds true or ot, with collocatios the whole is greater tha the sum of its parts. This idicates that the meaig of a collocatio caot be derived directly from the meaig of its parts. Either the meaig is completely differet from the free combiatio of its costituet words - as i the case of kick the bucket - or there is a added elemet of meaig to the whole phrase that caot be predicted from the parts - a characteristic also kow as o-compositioality (Maig ad Schütze, 1999). This complex meaig i collocatios costitutes a layer of sematics that is ot curretly cosidered by existig readability measures. This led us to propose collocatioal compoets as a factor i gaugig readability. Essetially, we should be able to ascertai a more accurate estimate for the sematic difficulty of a text by icludig its collocatios ad ot oly idividual words. 2. DERIVING THE AVERAGE COLLOCATION FREQUENCY I order to arrive at a practical applicatio for the sematic cotributio of collocatios we used the British Natioal Corpus (BNC) as a referece corpus. From this corpus we extracted, usig various tools ad techiques, collocatio frequecy lists. A review of such collocatio extractio tools is give i Aagostou ad Weir (2007b). The process we adopted for extractig collocatios was statistical i ature, as opposed to sytactical, ad was based o associatio measures to idetify co-occurreces (or collocatio cadidates). Such measures determie the stregth of bod betwee cotiguous words. Higher scores for a associatio measure, idicate greater probability that the co-occurrece extracted is a collocatio. Collocatio frequecy lists (CFLs) are similar i ature to word frequecy lists (WFLs), i which the two mai fields are the type of word ad its frequecy i the corpus - either as a percetage (relative frequecy) or as umber of occurreces (absolute frequecy). The key differece i CFLs, is that we measure the frequecy of collocatios rather tha idividual words. We propose that the frequecy of a particular collocatio i the CFL is a idicator of its sematic opacity, or simply put, its comprehesibility. This is a logical extesio of Zipf s Law ito the realm of multiword expressios: the higher the frequecy of occurrece for a collocatio, the easier it will be to uderstad. From this stadpoit, we formulated the Average Collocatio Frequecy. This is a factor of sematic difficulty that utilises CFLs ad as such is a departure from the philosophy of most readability formulae, which use a variat of word legth to determie sematic difficulty. The ACF is give by the followig equatio: ACF Where: m = ( 1 c) i= 1 fi i = f c + f acf = average collocatio frequecy c = total umber of collocatio occurreces i cosidered text m = umber of differet types of collocatios i cosidered text f i = frequecy of collocatio type i i referece corpus i = umber of occurreces of collocatio type i i sample text c + Κ + f m m c

3 The variable m is the umber of differet types of collocatio i the cosidered text. Note that by types of collocatio we do ot mea, for example, how may idiomatic phrases (Wermter ad Hah, 2004) or verb+ou costructs (Evert, 2004) are preset. Rather, the focus i this cotext is o how may differet istaces of collocatios exist i the text (see Figure 1 for a example ad further clarificatio). The ACF combies the absolute weight of a collocatioal toke i sematic opacity (based o a CFL) alog with the relative weight that it has iside the text (frequecy of a collocatio i the cosidered text). The absolute weight is represeted by the fi, which is the frequecy of collocatio type i (where 1<i<m) i the BNC - or whichever referece corpus is beig used for the extractio of the CFL. The CFLs produced from the BNC are the mai iput to the ACF ad they costitute the most crucial compoet of the measure, sice they serve as the comparator for ay cosidered text, ad determie the ACF s accuracy. The relative weight of a collocatio is represeted by the followig fractio, i/c, where i is the umber of times a collocatio of type i appears i the sample text ad c is the total umber of collocatioal tokes i the same text. The aforemetioed fractio appears whe expadig the summatio i calculatig the ACF, as depicted i the exteded form of the equatio ad i Figure 2. Cosider the followig sceario, usig the BNC Baby (approximately five millio words) as the referece corpus: The collocatio maa from heave appears oce i the BNC Baby, thus it has a frequecy of 0.2 occurreces per millio words (pmw). This is a rare collocatio, therefore it is difficult ad its impact o the sematic opacity of a passage that icludes it will be great. Furthermore, we take the view that the sematic impact of the collocatio is icreased if it appears more tha oce i a text. I other words, the higher the frequecy of occurrece of a difficult collocatio i a sample text, the harder uderstadig the text is goig to be. Ay measure of sematic difficulty based upo collocatios eeds to accommodate this fact, ad this is the role that the relative weight of a collocatio plays i the ACF. Figures 1 ad 2 illustrate a example of calculatig the ACF for a text i which three differet types (m=3) of collocatios have bee idetified (all values are hypothetical). Colum i icludes how may times each collocatio appears i the sample text. By addig the cells i this colum we ca calculate c, or the total umber of collocatio occurreces i the sample text. The cells i colum f i are populated with the frequecy of each collocatio i the referece corpus (CFL data), which, i this case, is measured as occurreces per millio words (pwm). Figure 1 provides all the data eeded to proceed to the calculatio of the ACF (Figure 2). umber of occurreces of collocatio type i i sample text m = 3 Collocatios foud i the sample text i f i put up with 1 = 2 f 1 = 60 pmw kick the bucket 2 = 1 f 2 = 22 pmw pull yourself together 3 = 4 f 3 = 46 pmw frequecy of collocatio type i i referece corpus umber of differet types of collocatios i sample text total umber of collocatio occurreces i sample text c = = = 7 Figure 1. Factorig Collocatios i the ACF

4 i i 7 i= acf = f = = = = 46.6 pmw pwm frequecy of collocatio put up with i the laguage relative weight of collocatio put up with i the sample text Frequecy of a hypothetical collocatio, which, should it substitute all the collocatios i the sample text, would have the same impact o sematic difficulty Figure 1. Derivig a value for the ACF As ca be see from Figure 2, the ACF acts like a replacemet or hypothetical collocatio, which, were it to substitute all the collocatios i the cosidered text, would have the same effect o the sematic difficulty of the text. I this example, the ACF is measured i occurreces per millio words, but that is dyamic. Due to the way the ACF is structured, its uit of measuremet is always the uit of the collocatio frequecies i the referece CFL. For istace, if the collocatio frequecies were percetages, the ACF would retur a percetage. 3. APPLICATIONS OF THE ACF Our primary iteded applicatio for the ACF is i readability formulae. As discussed i the previous sectios of this paper, existig formulae do ot iclude variables that tap the additioal levels of meaig embodied i collocatios ad do ot take advatage of the frequecy data ow available. The ACF should serve as a compoet i a readability formula ad thereby ehace the accuracy ad relevace of that readability measure. For example, a ew readability formula icorporatig the ACF could have the followig form: RF = a ACF + b WDI + c SDI + d Where: RF = readability formula (arbitrary title) ACF = Average Collocatio Frequecy (sematic difficulty idicator o the level of collocatios) WDI = Word Difficulty Idicator (commoly word legth) SDI = Sytactic Difficulty Idicator (commoly setece legth) a, b, c, d = coefficiets The additio of the ACF is the mai differece from most classical readability formulae ad, by accoutig for the collocatioal dimesio of sematics, promises a step forward i the accuracy of such tools. The WDI ad SDI compoets are geeric terms represetig the iclusio of difficulty idicators related to word cotet ad sytax, respectively. I order to complete the formula, its coefficiets eed to be calculated through regressio aalysis. Aagostou ad Weir (2007a) provide more detailed o the steps eeded to costruct a readability formula. The dummy formula above is a liear fuctio of its variables, but there are alterative formats as well. Bormuth (1969, cited by Klare, 1984) argued that curviliear formulae are more accurate i higher grade levels, whereas McLaughli (1969) suggested that the compoets of a formula should be multiplied istead of added.

5 We have argued that the ACF is a plausible igrediet i a readability measure. Furthermore, give the sematic complexity of may collocatios, i that the meaig of the whole is ot directly derivable from the meaig of the compoet terms, we cosider collocatios to be a sigificat hurdle to learers of Eglish. This will be especially so for ESL learers, sice they will lack the exposure to may collocatios that is available to ative learers of Eglish. So, i additio to its role as a factor i readability estimatio, we regard the ACF as a potet measure i its ow right ad evisage several specific applicatios i parallel to its use for readability. Recall that the ACF takes accout of the 'commoality' of collocatios (by referece to the frequecy of occurrece of collocatios i the BNC). Thereby, through the ACF, we gai a comparative measure of texts that allows us to rak such texts by their degree of collocatioal complexity. Such a rakig has potetial as a basis for gradig Eglish laguage learig materials, both for ative learers ad secod laguage learers of Eglish. I additio, the ACF affords a meas of matchig the collocatioal cotet of learig materials to the learig audiece (cf. Campbell & Weir, 2007). Sice secod laguage learers are more likely to lack exposure to the ordiary applicatio of collocatios tha ative speakers of Eglish, we cosider that the ACF will yield greatest beefit i this cotext. Maaged exposure to collocatios that icrease i complexity (i.e., are less frequet i their usage), promises to ehace the 'aturaless' of ay secod laguage learer's facility with Eglish laguage use. 4. SUMMARY AND CONCLUSIONS The ACF aims to quatify the impact of collocatios o sematic opacity. As such, it holds potetial ad opes up a ew path of research ito a layer of sematics curretly uaccouted for by traditioal complexity measures. Of course, some of our perspectives o collocatioal sematic impact may require further defese. For example, the ACF assumes that the repetitio of a difficult collocatio i a text further obfuscates its meaig ad makes it harder to uderstad. While we regard this as ituitive, it is possible that o occasios, the repetitio of a collocatio might lesse the effect it has o sematic opacity because the reader might be able to deduce its meaig (or part thereof) from cotext. I this case, the more ofte the collocatio appears, the higher the likelihood that its sematic opacity is reduced. While this may appear to coflict with the basis of the Average Collocatio Frequecy, we suggest that such cases are exceptios rather tha the orm. Ultimately, we may wish to accommodate such a possibility by gradually reducig the weight assiged to ay repeated collocatio. This is simply a variat that requires the empirical validatio that remais for us to address. A further couter may derive from earlier readability research to the effect that more complex predictors with more variables are ot ecessarily more powerful tha the simple oes (Bormuth, 1969, cited by Klare, 1984). I fact, as argued i Aagostou ad Weir (2007a), our approach does ot ivolve usig complex combiatios of multiple variables. Istead, we are proposig a simple idicator of sematic difficulty that embraces the wealth of meaig cotaied i collocatios. Furthermore, we stad to beefit from corpus data that were o-existet whe the aforemetioed research was coducted ad such data makes possible more accurate, laguage-level readability idicators, such as the ACF. The theoretical uderpiigs of the ACF lie i research ito readability, corpus liguistics ad collocatios. Further peer review ad testig are essetial to esure the measure s validity. A software applicatio that implemets the ACF is curretly i developmet. This will facilitate the required empirical testig. REFERENCES Aagostou, N. K. ad G. R. S. Weir. (2007a) From corpus-based collocatio frequecies to readability measure, i G. R. S. Weir & T. Ozasa (Eds), Texts, Textbooks ad Readability. Uiversity of Strathclyde Publishig, Glasgow, pp Aagostou, N. K. ad G. R. S. Weir. (2007b) Review of software collocatios for derivig collocatios, i G. R. S. Weir & T. Ozasa (Eds), Texts, Textbooks ad Readability. Uiversity of Strathclyde Publishig, Glasgow, pp Campbell, G. ad G. R. S. Weir. (2007) Matchig Readers to Texts with the Strathclyde Readability Measure, i G. R. S. Weir & T. Ozasa (Eds), Texts, Textbooks ad Readability. Uiversity of Strathclyde Publishig, Glasgow, pp

6 Dale, E. ad Chall, J.S. (1995). Readability Revisited: The ew Dale-Chall readability formula. Massachusetts: Brooklie Books DuBay, W.H. (2004). The Priciples of Readability. Costa Mesa, CA: Impact Iformatio. URL: Last Accessed 13 Jul Evert, S. (2004). The Statistics of Word Cooccurreces (Word Pairs ad Collocatios). Ph. D. dissertatio, Uiversitat Stuttgart. Firth, J. R. (1957). A syopsis of liguistic theory Studies i liguistic aalysis. The Philological Society, Oxford, pp Fraz, A. ad Brats, T. (2006). All Our N-gram are Belog to You URL: Last Accessed 11 Jul Klare, G.R. (1984). Readability. Hadbook of readig research, ed. P. D. Pearso. New York: Logma, pp Maig, C. D. ad Schütze, Hirich (1999). Foudatios of Statistical Natural Laguage Processig. MIT Press, Cambridge, MA. McLaughli, G.H. (1969). SMOG Gradig A New Readability Formula. Joural of Readig, 22, pp Steer, A. J., I. Horabi, Smith, D. R., Smith, R. (1988). The Lexile Framework. Durham. NC: Metametrics, Ic. Wermter, J. ad Hah. U. (2004) Collocatio extractio based o modifiability statistics. I, Proceedigs of the 20th Iteratioal Coferece o Computatioal Liguistics, Geeva, Switzerlad Zipf, George Kigsley (1949). Huma Behavior ad the Priciple of Least Effort. Addiso-Wesley, Cambridge, MA.