The fusion of meteorological- and air quality information for orchestrated services using environmental profiling

Transcription

1 The fusion of meteorological- and air quality information for orchestrated services using environmental profiling Lasse Johansson, Ari Karppinen Finnish Meteorological Institute Department of air quality modelling Helsinki, Finland Leo Wanner Dept. of Information and Communication Technologies Pompeu Fabra University Barcelona, Spain Abstract Citizens are increasingly aware of the influence of environmental and meteorological conditions on the quality of their life. Thus, there has been a growing demand for personalized information about air quality and weather, i.e., information that is tailored to citizens' specific context and background. A recent poll showed that in almost all EU member states the citizens would like to have easier access to air quality related information. To address this demand, an orchestrated web service (PESCaDO) has been developed. The PESCaDO system is designed to discover reliable environmental data in the web, to process these data in order to convert them into knowledge and to use this knowledge for the provision of human-centered information. In this paper, a description of the abovementioned process is given focusing on the fusion of environmental information into a coherent data block. We will show that with the presented methods, superior temperature forecasts with respect to individual service providers can be provided to the users of PESCaDO. Furthermore, by utilizing a land use mask, a population density mask and an array of historical measurements at various locations, the regionally calibrated fusion method can be used to produce accurate air pollutant concentration maps on a local scale. In particular, we will show that the hourly average NO2 concentration in a remote city center can be predicted accurately using measurements that are no less than 50 kilometers away from the location. Keywords environmental information fusion, service orchestration, PESCaDO portal. I. THE PESCADO SERVICE AND METHODOLOGY In recent years, the emergence of social media, personalized web services and an increased public awareness of environmental factors that impact the quality of life have led to the demand for easier access to environmental information and it s tailoring to personal needs. Focusing on the atmospheric environment alone, there is a need for an integrated assessment of the impact of air pollution, allergens and extreme meteorological conditions on public health, as well as for the provision of related information to citizens [5], [6]. Yet, getting a direct answer to seemingly simple questions like How will the air quality be tomorrow in Helsinki? involves a lot of manual search and interpretation of the often contrasting information found on various web sites. In order to formulate accurate answers regarding air quality and atmospheric conditions, information from a large number of online sources are needed to be searched for relevant input data. The quality of information however might vary significantly in reliability and relevance with respect to to the location and time of interest. On the other hand, even biased and inaccurate information about air quality could be utilized with the help of data ensembling methods. Indeed, the socalled multi-model approaches to ensemble forecasting are receiving significant attention. Justification for the approach rests on the apparent lack of divergence of model solutions over short time ranges [7]. The success of multi-model ensemble combination has been demonstrated in many studies given that information about all participating models, including the less skillful ones is available [4]. Approaches to fuse measured air quality information in a large geographical scale using air quality data from multiple sources have been presented, such as the statistical air pollution model RIO [8]. However, fusion approach such as [8] is difficult to utilize while the structure and nature of input data is extremely heterogeneous containing simultaneously model forecasts, station measurements with varying reliability, time of validity and location. Spatial and temporal gaps are also a matter of concern; measurement stations are almost always sparsely located and small in numbers, while even dense coverage models have a finite spatial-temporal resolution. These considerations in turn lead to the need to use some form of data interpolation either in space or time domain, or both. Ultimately, there needs to be a universal way to measure and compare the quality of information with respect to source reliability, spatial and temporal separation with respect to certain location and time.

2 In this paper, an online service called PESCaDO is presented. The general architecture of the PESCaDO system, which aims to provide EU citizens with accurate and timely information about local air quality and weather conditions, is described focusing on the after-treatment of extracted information. Also, a generalized method for the fusion of processed meteorological and air quality data is presented. In this context we define fusion to be intelligent assimilation of data, with respect to specified location and time, from heterogeneous data sources describing the same phenomenon by utilizing environmental profiling and reliability metrics for individual data sources. In this sense fusion can be described as piecewise ensembling while accounting for spatial-temporal differences. A brief evaluation of fusion performance is presented in two different setups: i) the fusion of local measured NO2 concentration and ii) the fusion of temperature forecasts (4 service providers) in multiple locations. II. PESCADO SERVICE AND METHODOLOGY The purpose of PESCaDO is to address the need for timely environmental information, by (i) taking as a starting point plain text queries in natural languages or interactive user input through a web interface, in a way that transcends simple keyword-based searches (ii) infer the implied context and semantic meaning of the user's query (e.g., place, time, activity, type of information desired) by using advanced semantic and ontological textual analysis tools, and (iii) coordinate the data flow from a number of heterogeneous (text, images, feeds, binary files) data sources. Ultimately, a response is produced and presented to the user by taking into account all available online data, as described in [2] and [3]. During the development of PESCaDO, a service oriented approach (SOA) was selected and thus we refer to several independent services in this paper. Fig 1: Schematic diagram of the work flow of the PESCaDO system. A. Work-flow in a nutshell The processing of information (see Fig. 1) begins when the fusion service (FS) is given a request to produce fused information about a selected set of relevant aspects, such as NO2 concentration and temperature, tomorrow between 12:00 and 18:00, in a specified region in Helsinki. The user profile and selected activities (i.e. hiking ) affects the set of related aspects given to FS. Furthermore, the user profile (administrative/citizen) affects the way the response is ultimately presented to the user. The Data Retrieval Service (DRS) serves as an interface, through which other services can retrieve the information. FS queries DRS to provide environmental data reported for requested geographic areas, time periods for all related aspects. DRS also provides FS with supporting information needed in the fusion process, such as node identification and coordinates if available, and node reliability information. Some of the retrieved information may be in text form, for instance, instead of a numerical value describing rain in mm/h a piece of information may be a simple text heavy rain. Such text responses are converted to numerical form with the help of PESCaDO s ontology: the Knowledge Base is queried about the upper and lower limit for heavy rain in the specified region and then the average value of the returned limits can be taken to represent the input in numeric form. Once all the input data is in numeric form FS fuses the data one aspect at a time, utilizing available uncertainty metrics for each information source given by the uncertainty metrics tool (UMT). The uncertainty metrics tool works individually under FS by storing measurements and forecasted data and gradually evaluates the prediction accuracy of environmental nodes locally. Using this information UMT evaluates measured values against forecasts autonomously and produces updated source node uncertainty metrics. After fusion the processed information is stored in the Knowledge Base (KB) after which the Answer Service (AS) is notified to go on with the workflow. Relevant information is selected, structured and presented to the user in an easily understandable form. III. FUSION OF EXTRACTED INFORMATION The fusion of information in an orchestrated service such as PESCaDO offers the user several advantages: Only the information relevant to the original request can be presented rather than a collection of relevant extracted information. Secondly, a large number of independent data describing the same phenomenon can be combined in a way that the fusion result will be of superior quality with respect to the individual sources [1]. Furthermore, small geographic and temporal gaps in the input data can be extrapolated if needed. Information about air pollutant concentrations and weather conditions loses accuracy rapidly as a function of its age and distance, since atmospheric conditions are ephemeral and volatile phenomena. Individual pieces of information from different nodes can seldom be regarded as equally relevant and thus a general measure for information relevance and quality is needed for data fusion. In the fusion process, all pieces of meteorological and air quality data reflect conditions of certain time and place. These pieces of information are regarded as statistical estimators for the conditions in the user defined area and time, where (1)

3 and is the error in terms of statistical variance and bias. In this respect, the relevance and quality of information are measured with statistical methods. The fusion service estimates an aggregate statistical variance measure for each piece of information and the derived imprecision metrics are then used for the assignment of averaging weights to each input dataset. Essentially a large estimated aggregate imprecision measure causes the assigned weight to decrease while the data from the more accurate and relevant sources are assigned larger weights and, thus, gain more emphasis in the fusion. A. Variance estimation The aggregate variance measure aims to quantify, for instance, what is the expected precision in terms of variance between a pollutant concentration at time t 0 in location r 0 - given by a certain environmental node (e.g. a model or a measurement station) - and another pollutant concentration in time t and location r. A schematic diagram of this total imprecision estimation process is presented in Fig. 2. temporal component, the achieved correlation of these polynomial variance models is generally very high (R 2 > 0.95). For selected pollutants and temperature, these estimated polynomial models have been presented in Fig 3. Figure 3: Temporal polynomial models (auto-covariance plots) as function of measurement age for selected variables used in total variance estimation. For viewing pleasure, temperature and PM2.5 curves has been amplified. Fig 2: Schematic diagram of the total imprecision model used in the fusion service. The statistical variance measures for (i) temporal separation component, (ii) spatial separation component and (iii) node base variance component is estimated separately. For the latter, the UMT (see Fig 1) which has stored information about data node s prediction accuracy in the past is utilized. Naturally, in case of measurement nodes the historical performance of a measurement station is negligible. We assume that the three individual components described in the variance model are statistically independent and thus the aggregate variance is simply the sum of their individual component s variances. For the calibration of temporal and spatial components, historical measurement time series was gathered during 2010 from stations depending on pollutant type across Finland. To facilitate the use of fusion service in other regions than Finland however, the calibration of aggregate variance model parameters can be re-evaluated with existing processes with any set of measurement time series. Especially with the In contrast to temporal component, the distance component evaluation is significantly more difficult than the temporal component, resulting in polynomial model with more modest correlation coefficients. The difficulties arise from fact that different locations may not be equal in terms of pollutant concentrations to facilitate such direct statistical analysis. Another complication related to the distance evaluation in the PESCaDO framework arises from the fact that sometimes only an approximated location for the piece of information can be given; this is usually the case with extracted weather forecasts for cities. In these cases the city center coordinates are used while the additional inaccuracy arising from the missing coordinates can be taken into account within the distance variance component by applying a distance penalty comparable to the radius of the city. B. Bias correction In PESCaDO, the user query specifies an area of interest for which the resulting response will relate to. Local measurements given by meteorological and air quality measurement stations which are used as input data however, may not relate well with the specified area. For instance, if the user queries information for a natural park just outside an urban area the input data might very well contain measurements describing local urban air quality. Given that the average pollutant concentrations are typically clearly higher in an urban environment than in a more rural outskirts area, the estimators may incorporate significant amount of bias. More importantly, the derived aggregate variance estimate that quantifies precision cannot account for such inaccuracy.

4 In order to detect and reduce the bias from estimator due to such geographical incompatibility, the fusion service utilizes a geographic profiling feature using CORINE land-use data in conjunction with a population density 2010 data. Based on the surrounding land use (with small evaluation radius) and local population (a wider evaluation radius) the expected hourly average concentration of a selected pollutant can be evaluated (Fig. 4). Fig 4: Profile evaluation with land use and population density maps. The larger circle represents the area for local population determination and the smaller red circle represents the area for land use determination. The profiling feature of the fusion service estimates the expected concentration of an air pollutant as a function of population size and land use frequencies in a specified area. The local population allows the detection of cities with variable sizes while the land use information allows different kind of regions inside the city to be identified. Furthermore, the population count yields the density and land use gives us the volume of a factor, which in turn can be multiplied with each other to get sound variables for multi-variable linear regression. More specifically, we estimate the hourly excepted pollutant concentration as follows: Where θ(r,h) is the pollutant concentration in location r in the hour of day h while the location r exhibits the land use frequencies f S, f U, etc. p is the population count in the surrounding area, f S is the relative frequency of suburban landuse, f U is the relative frequency of urban land-use, f ei is economic and/or industrial land-use, f T for roads and c is constant. For the evaluation of parameters a 1 a 5 the following process is undertaken: - A large collection of air pollutant measurements from various environments is taken. - For all measurement sites, the average pollutant concentration for each hour of day (individually for working days, Sundays, summer, winter etc.) is calculated from the time series. - The environmental profile (population density and land use frequencies) for each site is evaluated (2) - For each hour of day (individually for working days, Sundays, summer, winter etc.), a multi-variable linear regression for Eq. 3 is evaluated with ordinary least squares method (OLS). - The resulting optimal parameterization of a 1 a 5 in Eq.3 are stored and used in fusion. The calibration of aggregate variance model has been automatized inside the fusion service so that the PESCaDO system can be easily adapted and used in any country. Hence, the abovementioned calibration of environmental profiling was designed and implemented so that the evaluation of profiling parameters can be done automatically using the same set of evaluation measurements. By comparing the profiles of the target location and the specified coordinates of the estimator, it is possible to reduce the bias resulting from the differences between the expected hourly averages. Let p i (r,t) be the estimator profile and p 0 (r 0,t 0 ) be the evaluated profile corresponding to the user defined location and time. Then, a bias corrected estimator is given by where is the expected hourly concentration of the pollutant near the specified location r in time t and is the expected pollutant concentration in the target location and time, respectively. As an example, In Fig 5a-d the expected and observed hourly average of NO 2 concentrations for four locations/environments in Finland is presented. Fig 5a-d: Measured and predicted hourly NO2 concentrations for 4 different measurement sites. Predicted values have been calculated with multi-variable regression models, which were derived without the four stations being included. Evaluation period: January to May 2011 in Southern Finland. C. Optimal weight calculation Assuming all data source to be independent and non-biased (after bias removal), an optimal ensemble value using statistical total variance estimates can be calculated according to [1] given by: (3)

5 where individual weights w i is given by (4) To assure statistical independence, only a single estimator per node is taken for fusion calculation. The value selected is simply the one with the lowest aggregate variance; for any fixed measurement station time series the value which has the closest time of validity with respect to the user defined time is selected as the base and distance variance components are the same for all measurements for that station. Theoretically, it can be shown that the fused ensemble value is the optimal estimator in terms of mean squared error and that the prediction accuracy increases while the number of independent data sources (n) is increased [1]. More importantly, does not suffer from very low quality estimators as long as reasonably accurate imprecision metrics is used. IV. RESULTS The performance of the presented environmental information fusion method was evaluated using temperature forecasts provided by four well known Scandinavian weather service providers (SP). For 43 locations around Finland weather forecasts were extracted from respective online sites and stored during several months in Uncertainty metrics in terms of MSE for individual SPs were evaluated by comparing measured temperature values against individual stored forecasts for each SP; a total of 2500 forecasted temperature versus measured temperature -pairs for each SP was gathered in order to get statistically meaningful MSE estimates as a function forecast period length. Then, fused forecasts (temperature of the next 3 days) for the locations in August 2012 were produced on a daily basis for each of these locations using the stored forecasts. (5) locations and time periods in august was used. Long term forecasts from SP1 was not available. In Fig. 6, the mean absolute error of temperature forecasts and the fused forecast is presented. Naturally, MSE increases as a function of forecast length. According to the figure fused temperature forecasts have the lowest mean error compared to any individual SP with just four different SPs providing forecasts simultaneously. A. Fusion of measured NO 2 hourly concentration The performance of the fusion of air quality measurements with the presented methodology was tested with NO 2 measurements in Southern Finland. Measurement time series for February in 2010 from several stations was used as input data and fused NO 2 concentrations were calculated for another location at the center of a selected small city. The locations of measurement stations and the evaluation city have been marked in Fig. 7. Then, the fused hourly estimates in the selected city center were compared against on-site measurements (Fig. 8). The selected test corresponds to one of the PESCaDO use cases where an administrative civil engineer of a remote city queries information about local air quality without having their own measurement network established. Fig 7: Fused NO2 concentration in Southern Finland, in February 1st 2010 at 07:00, is shown. Yellow markers represent measurement stations providing the input data for the fusion. Yellow star indicates the location for fused time series presented in Fig 8. Background satellite image is provided by (Google Earth, 2012). Fig 6: Mean absolute error of temperature forecasts and the fused forecast for different forecast time spans. Forecasted and measured data for 43 different

6 Fig 8: Fused and observed time series of fused NO2 concentration in a small city centre (yellow star marker at Figure 5) in February Input data for the fusion was provided by the measurement stations displayed in Fig 6 Onsite measurements were not used as input data in the fusion. In Fig. 7 it can be seen that the highest concentrations of NO 2 can be found at the center of the largest metropolitan area in Finland. Due to the environmental profiling, the high measured value at the center of metropolitan area will not cause high concentration values to be extrapolated to near-by locations with signficantly less amount of road traffic and urbanization. The comparison between fused and measured NO 2 concentration at the test site in Fig. 8 shows that the pollutant concentration has been estimated fairly accurately with the presented method. During the study period the mean error of estimation is approximately 9 µg/m3. Furthermore, the same fusion comparison was performed without the environmental profiling and the resulting mean error was approximately 17µg/m3. V. CONCLUSIONS In order to provide timely meteorological and air quality related information to citizens and administrative users alike, a prototype service PESCaDO has been developed. For the specific needs of this system a fusion service was designed and implemented to account for the intelligent assimilation of conforming and competing data from different environmental nodes. With the fusion method described in this paper it possible to produce accurate information to the users of this service in the future. The service itself was designed to be self-maintaining and learning; as more and more information flows through the system the imprecision metrics for individual nodes can be evaluated continuously. Ultimately, the various environmental parameters that the fusion service needs for accurate fusion results can be automatically evaluated and stored. It was shown that the fusion of temperature forecasts for cities can be performed so that the prediction accuracy is equal or even lower than the best performing service providers. However, this result should be further investigated using a longer evaluation period and also, with other weather variables such as wind speed and precipitation. Fusion of measured NO 2 concentrations by using environmental profiling showed promising results. The hourly concentration in a the selected remote city center was predicted with high accuracy even though the nearest measurement site was more than 50 kilometers away and in a totally different environment. It should be noted that of all the pollutants accounted for in PESCaDO, NO 2 has the strongest dependency to land use type and population density in Finland. Depending on the hour of day, the regression coefficient R 2 for the expected NO 2 concentration given by Eq. 2 is between 0.7 and For PM 2.5 concentration, which has a stronger health effect on the population, the benefits from environmental profiling are more modest. However, the presented environmental profiling offers several advantages that could be further exploited. For instance, the range of monitored land use types could be expanded to include different types of forests and this information could be used in the fusion of pollen concentration. Furthermore, location of harbors can be identified from land use data while large harbor areas are a major source of some specific air pollutants like SO 2. Also, the environmental profiling feature could be expanded with topography and more accurate traffic-flow information. ACKNOWLEDGMENT The research leading to these results has received funding from the project EU PESCaDO (FP7-ICT ) REFERENCES [1] Potempski, S. and Galmarini, S., Est modus in rebus: analytical properties of multi-model ensembles, Atmos. Chem. Phys., 9, , doi: /acp , [2] Wanner, L., Bosch, H., Bouayad-Agha, N., Bügel, U., Casamayor, G., Ertl, T., Karppinen, A., Kompatsiaris, I., Koskentalo, T., Mille, S., Mossgraber, J., Moumtzidou, A., Myllynen, M., Pianta, E., Rospocher, M., Saggion, H., Serafini, L., Tarvainen, V., Tonelli, S., Usländer, T., and Vrochidis, S., Service-Based Infrastructurs for User-Oriented Environmental Information Delivery, Proceedings of the Enviroinfo Workshop on Environmental Information Systems and Services Infrastructure and Platforms. Bonn, Germany ( 679/), 2010 [3] Wanner, L., Vrochidis, S., Tonelli, S., Mossgraber, J., Bosch, H., Karppinen, A., Myllynen, M., Rospocher, M., Bouayad-Agha, N., Bügel, U., Casamayo,r G., Ertl, T., Kompatsiaris, I., Koskentalo, T., Mille, S., Moumtzidou, A., Pianta, E., Saggion, H., Serafini, L., and Tarvainen, V,. Building an Environmental Information System for Personalized Content Delivery. In (Hrebícek J., Schimak G., Denzer R. eds.): Environmental Software Systems. Frameworks of eenvironment - 9th IFIP WG 5.11 International Symposium, Proceedings. IFIP Publications 359, Springer, ISBN , pp , [4] Weigel, A.P, Liniger, M.A and Appenzeller, C Can multi-model combination really enhance the prediction skill of probabilistic ensemble forecasts?. QUARTERLY JOURNAL OF THE ROYAL METEOROLOGICAL SOCIETY. Q. J. R. Meteorol. Soc. 134: (2008) [5] Κlein Τh., Kukkonen J., Dahl Å., Bossioli E., Baklanov A., Fahre Vik Α., Agnew P., Karatzas, K., and Sofiev, M., Interactions of physical, chemical and biological weather calling for an integrated assessment, forecasting and communication of air quality, AMBIO, 2012, accepted.

7 [6] Karatzas, K. and Kukkonen, J., COST Action ES0602: Quality of life information services towards a sustainable society for the atmospheric environment, ISBN: , Thessaloniki: Sofia Publishers, [7] Lewis, M. Roots of Ensemble Forecasting. American meteorological Society,July 2005, p [8] Janssen, S., Dumont, G., Fierens, F. and Mensink, C. Spatial interpolation of air pollurion measurement using CORINE land cover data. Atmospheric Environment, Vol. 42, Issue 20, June 2008, p