EVALUATION OF DATA QUALITY IN LICHEN BIOMONITORING STUDIES: THE ITALIAN EXPERIENCE. 1. Introduction

Transcription

1 EVALUATION OF DATA QUALITY IN LICHEN BIOMONITORING STUDIES: THE ITALIAN EXPERIENCE GIORGIO BRUNIALTI 1, PAOLO GIORDANI 1, DEBORAH ISOCRONO 2 and STEFANO LOPPI 3 1 DIP.TE.RIS, University of Genova, C.so Dogali 1 m, Genova, Italy; 2 Department of Plant Biology, University of Torino, Torino, Italy; 3 Department of Environmental Sciences, University of Siena, Siena, Italy ( author for correspondence, loppi@unisi.it) (Received 28 November 2000; accepted 2 May 2001) Abstract. A total of 65 operators involved in lichen mapping studies in central and northwestern Italy underwent quality control tests during five lichen biomonitoring workshops organized between 1999 and The results showed that 75% quantitative accuracy and 90% quantitative precision can be regarded as satisfactory levels for lichen biodiversity data; 65% proved to be sufficient for accuracy of taxonomic identification in the field. Average correct assignment of the interpretative naturality/alteration class was only 48.7%. The results indicated the need for taxonomic training. Keywords: biomonitoring, data quality, Italy, lichen biodiversity, quality control, quality evaluation 1. Introduction The variability of biological data often affects the forecasting precision of ecological approaches to the evaluation of environmental quality (Laskowsky and Kutz, 1998). The quality of the data from biological measurements depends heavily on at least three factors (Kovács, 1992): variability of the biomonitor organisms (interactions between the organism and environmental factors), type of sampling (sampling design, density of sampling points), operators involved in data collection (especially for methods requiring taxonomic knowledge). Although many lichen biomonitoring surveys have been carried out in Italy in the last ten years (Piervittori, 1999), two main obstacles have prevented mapping of the whole country: high phytoclimatic heterogeneity and the lack of a standardized protocol. The use of lichens in mapping studies is based on the assumption that ecological parameters other than air pollution have a constant effect on these organisms (Richardson, 1988). Nevertheless, it is widely accepted that climate and other factors affect lichen communities (Barkman, 1958) and as a consequence it is difficult to discriminate the effect of air pollution on the frequency and distribution of lichen species from the effects of other environmental parameters (De Wit, 1976). To overcome this problem, different scales for the interpretation of lichen Environmental Monitoring and Assessment 75: , Kluwer Academic Publishers. Printed in the Netherlands.

2 272 G. BRUNIALTI ET AL. TABLE I Data regarding the quality control tests in five places in Italy Elba Island Rapallo Peveragno S. Stefano Montecatini d Aveto Terme No. of operators No. of sampled trees Range of LB values Mean LB value ± SD 89.5± ± ± ±7.4 57±13.5 biodiversity values obtained in different phytoclimatic macroregions of Italy are being studied (Loppi et al., 2001). In order to provide a standardized sampling protocol for data collection, Italian guidelines for monitoring the effects of air pollution by phytotoxic gases (especially SO 2 and NO x ) using the biodiversity of epiphytic lichens have recently been published (Nimis, 1999). However, in the implementation of the guidelines in a large-scale approach, the documentation of standard operating procedures and a proper sampling design are only the first steps in the evaluation of data quality (Cline and Burkman, 1989). To our knowledge, the training, calibration and control phases (quality control, QC) and statistical evaluation of data quality (quality evaluation, QE) have not been addressed in Italy. To assess the reliability and consistency of data collected in large scale surveys, where field activities are generally carried out by non-specialists, two activities are fundamental: training of the operators involved in data collection and field control on data reproducibility (Edwards, 1998; Ferretti, 1997). In the last two years, several lichen biomonitoring workshops and courses have been organized by Regional Agencies for Environmental Protection (ARPA) in Tuscany, Piedmont and Liguria. The main goals of these meetings were to train the operators involved in mapping studies and to compare the lichen biodiversity (LB) counts measured by different operators from different parts of Italy. Here we report the results of QC and QE regarding the influence of the operators on data quality in lichen biomonitoring surveys, performed during these workshops. 2. Materials and Methods A total of 65 operators (note that the grand total of Table I is 78 because some operators repeated the test after seven months, see further) involved in lichen mapping studies in central and northwestern Italy underwent QC tests during five lichen biomonitoring workshops organized in Italy between 1999 and 2000: 1) Elba Island,

3 DATA QUALITY IN LICHEN BIOMONITORING STUDIES 273 Tuscany, May 1999; 2) Rapallo, Liguria, October 1999; 3) Peveragno, Piedmont, April 2000; 4) S. Stefano d Aveto, Liguria, May 2000; 5) Montecatini Terme, Tuscany, September 2000 (Table I). In line with the Italian guidelines (Nimis, 1999), LB was measured as the sum of frequencies of epiphytic lichens in a sampling grid of cm divided into 10 units of cm. The bottom of this grid was placed on the trunk of standard trees at a height of cm from the ground, on the part of the bole with the highest lichen abundance. All the lichen species within the grid were noted together with their frequency, namely the number of grid units in which the species was present. The sum of the frequencies of all species was the LB of the tree QUALITY CONTROL The sampling grids were placed by a group of expert lichenologists on the bole of selected trees. Each operator was allowed about 30 min to sample the trees without any diagnostic key or chemical test. To check the precision of the data (see below), one of the trees was re-sampled. The results of each operator were compared with the scores of the group of experts (control) QUALITY EVALUATION Two main procedures of QE of the data, accuracy and precision, were used to assess the percentage error of LB counts (Tallent-Halsell, 1994). The accuracy is the percentage deviation of the data collected by the operators from the control. It is calculated: accuracy % = 100 [100 (1 op/exp)], where op is the count scored by the operator and exp the count scored by the group of experts on the same tree. The precision of the data is the percentage reproducibility of the same sampling by the same operator. It is calculated: precision % = 100 [100 (1 op min /op max )], where op min and op max are the minimum and maximum values scored by an operator in two separate samplings on the same tree, respectively. Three types of accuracy/precision measures were considered: 1) quantitative, to evaluate the LB counts; 2) qualitative, to evaluate the number of lichen taxa recognized in the grid; 3) taxonomic, to evaluate the correct identification of lichen taxa in the grid. To evaluate the percentage of satisfactory trained operators, the measurement quality objectives (MQOs) were set aprioriat 75% for accuracy and 90% for precision. 3. Results and Discussion In general, QC tests were carried out at the end of a one-week lichen biomonitoring course, so most operators had little knowledge of lichen identification. As a

4 274 G. BRUNIALTI ET AL. Figure 1. Percentage accuracy of lichen biodiversity counts in quality control tests in five places in Italy. The horizontal line represents the 75% measurement quality objective (MQO). consequence, localities with the highest lichen diversity (Elba and Rapallo) resulted more hard, as demonstrated by the highest coefficients of variation (data not shown). The average accuracy of LB counts (Figure 1) ranged from 51.8% to 88.1%, but the MQO was only reached in two tests (Peveragno and Aveto), while in the other places only a few operators (Elba) or no operator at all (Rapallo and Montecatini) scored a satisfactory result. Despite this, the present results are in line with the average accuracy (56.1 ± 6.9%) obtained in similar tests of visual assessment of leaf damage injury induced by ozone in indicator tobacco plants (Lorenzini et al., 2000). The MQO for accuracy on the number of lichen taxa (Figure 2) was reached in the Aveto and Peveragno tests (80.6 ± 9.1% and 81.7 ± 11.4%, respectively); few operators reached it at Elba (66.0 ± 11.1%), Rapallo (54.3 ± 12%) and Montecatini (66.9 ± 8.7%). These results can be explained by the lack of field experience of the operators, who could not distinguish similar species. The average percentage of accuracy in taxonomic identification was rather low (Figure 3), ranging from 39.4 to 55.4%. The MQO was not reached in any test. However, these results are probably underestimated because the operators had to identify the species directly in the field without any diagnostic key or chemical test, while normally, species that cannot be identified in the field are collected and determined in the laboratory. The results and this fact suggest that the MQO for

5 DATA QUALITY IN LICHEN BIOMONITORING STUDIES 275 Figure 2. Percentage accuracy of number of lichen taxa recognized in quality control tests in five places in Italy. The horizontal line represents the 75% MQO. Figure 3. Percentage accuracy of taxonomic identification in quality control tests in five places in Italy. The horizontal line represents the 75% MQO.

6 276 G. BRUNIALTI ET AL. Figure 4. Percentage accuracy recalculated after grouping operators into three classes of lichenological experience. The horizontal line represents the 75% MQO. accuracy of taxonomic identification should probably be lowered to 65%, which is still a good identification level in the field (McCune et al., 1997). The level of lichenological knowledge of the personnel checked during the QC tests was very uneven and mean accuracy was recalculated after grouping the operators into three classes of lichenological experience: low, medium and high (Figure 4). Accuracy of taxonomic identification differed between groups, ranging from 28.9% (low experience) to 61.2% (high experience). It is noteworthy that the MQO for accuracy of taxonomic identification was not met in any case, even at 65%. The MQO for LB and qualitative accuracy (75%) was only reached by the groups with medium or high lichenological experience, suggesting that the average quality of the data produced by the operators can be improved. Some of the operators repeated the QC test seven months later, after much fieldwork (Figure 5), and showed a great improvement in data quality, both quantitative LB accuracy (from 72.4 to 84.6%) and accuracy of taxonomic identification (from 32.1 to 56.1%). Operators often improved in accuracy during the same test, and the present results confirmed that accuracy improved with taxonomic training and, above all, continuous fieldwork. In general, precision was very high for LB counts, number of species in the sampling grid and, especially, taxonomic identification (Figure 6). These results suggest that lichen biodiversity counts by the operators had a high reproducibility, which is extremely important for correct evaluation of time-series in biomonitoring

7 DATA QUALITY IN LICHEN BIOMONITORING STUDIES 277 Figure 5. Percentage accuracy of lichen biodiversity counts (LB), number of lichen species (NS), and taxonomic identification (TI), before (1) and after (2) a 7-month training period. Figure 6. Percentage precision for lichen biodiversity counts, number of species recorded and taxonomic identification skill.

8 278 G. BRUNIALTI ET AL. Figure 7. Average correct assignment of the interpretative naturality/alteration class (a) and deviation from the control (b). studies. However, it should be borne in mind that changes in operators in long-term monitoring of permanent plots can give misleading results (McCune et al., 1997). As LB data are generally interpreted in terms of deviations from natural conditions on a well defined scale (Table II, Loppi et al., 2001), the deviation from the interpretative classes of the LB values obtained by each operator was also checked (Figure 7). Average correct assignment was only 48.7%; deviation from reality (control) was almost one class below (35.4%) and two classes below (10.1%) for Elba and Rapallo, i.e. at localities with the highest lichen diversity. In a few cases (5.8%) the bias was positive and the class obtained was above control.

9 DATA QUALITY IN LICHEN BIOMONITORING STUDIES 279 TABLE II Scale for the interpretation of LB values on Tilia and deciduous Quercus trees from the Tyrrhenian side of Italy (Loppi et al., 2001) LB values % Deviation from Interpretation natural conditions Lichen desert Alteration Semi-alteration Semi-naturality > Naturality 4. Conclusions In order to estimate the limits of a lichen biomonitoring survey and to provide the bases for further improvement, along with the results of the survey itself, there is the need for reporting also the evaluation of data quality (Shampine, 1993). However, it is difficult to establish MQOs apriorifor a survey. Our data indicate that 75% quantitative accuracy and 90% quantitative precision are satisfactory levels for lichen biodiversity data and 65% is sufficient for accuracy of taxonomic identification in the field. The present study is the first application of large-scale QC tests in lichen biomonitoring studies in Italy. However, these QC tests only focused on the last phase of a LB sampling protocol (i.e. the calculation of lichen frequencies in the sampling grid). In order to check the variability of LB data at each step of sampling, other aspects of QC, such as the selection of sampling trees, should also be evaluated. Acknowledgements We thank all the operators who performed the QC tests. This study was co-financed by MURST funds, project Cryptogams as Biomonitors in Terrestrial Ecosystems. References Barkman, J. J.: 1958, Phytosociology and Ecology of Cryptogamic Epiphytes, VanGorcum, Assen. Cline, S. P. and Burkman, W. G.: 1989, The Role of Quality Assurance in Ecological Programs. In J. B. Bucher and J. Bucher-Wallin (eds), Air Pollution and Forest Decline, Iufro, Birsmendorf, pp De Wit, T.: 1976, Epiphytic lichens and air pollution in The Netherlands, Bibl. Lichenol. 5,

10 280 G. BRUNIALTI ET AL. Edwards, D.: 1998, Issues and themes for natural resources trend and change detection, Ecol. Appl. 8, Ferretti, M.: 1997, Forest health assessment and monitoring: Issues for consideration, Environ. Monit. Assess. 48, Kovács, M.: 1992, Biological Indicators in Environmental Protection, Horwood, New York. Laskowsky, S. L. and Kutz, F. W.: 1998, Environmental data in decision making in EPA regional offices, Environ. Monit. Assess. 51, Loppi, S., Giordani, P., Brunialti, G., Isocrono, D. and Piervittori, R.: 2001, A new scale for the interpretation of lichen biodiversity values in the Tyrrhenian side of Italy, Bibl. Lichenol. (in press). Lorenzini, G., Nali, C., Dota, M. R. and Martorana, F.: 2000, Visual assessment of foliar injury induced by ozone on indicator tobacco plants, Environ. Monit. Assess. 62, McCune, B., Dey, J. P., Peck, J. E., Cassell, D., Heiman, K., Will-Wolf, S. and Neitlich, P. N.: 1997, Repeatability of community data: species richness versus gradient scores in large-scale lichen studies, Bryologist 100, Nimis, P. L.: 1999, Linee-guida per la Bioindicazione Degli Effetti dell Inquinamento Tramite la Biodiversitá dei Licheni Epifiti. In: C. Piccini and S. Salvati (eds), Proc. Workshop Biomonitoraggio della Qualità dell Aria Sul Territorio Nazionale, Roma, November 1998, ANPA, Roma, pp Piervittori, R.: 1999, Licheni come Bioindicatori della Qualitá dell Aria: Stato dell Arte in Italia. In: C. Piccini and S. Salvati (eds), Proc. Workshop Biomonitoraggio della Qualitá dell Aria Sul Territorio Nazionale, Roma, November 1998, ANPA, Roma, pp Richardson, D. H. S.: 1988, Understanding the pollution sensitivity of lichens, Bot. J. Linn. Soc. 96, Shampine, W. J.: 1993, Quality assurance and quality control in monitoring programs, Environ. Monit. Assess. 26, Tallent-Halsell, N. G.: 1994, Forest Health Monitoring. Field Methods Guide, EPA/620/R-94/027. U.S. Environmental Protection Agency, Washington D.C.