Alberta Real-time Ambient Air Data Submitter s Guide

Transcription

1 Title: Number: Program Name: Air Policy Original Release Date: December 19, 2017 Effective Date: January 1, 2019 This document was updated on: Overview Air quality data (and meteorological data) collected at continuous ambient air monitoring stations is required to be submitted electronically in real-time to the Government of Alberta as per the Air Monitoring Directive (AMD Chapter 9, Section 13, as amended from time to time). This guide sets out the methods and technical specifications for transmitting that data, as well as required and optional parameters, the units in which each parameter must be transmitted, file formatting, server connectivity, and timing requirements. The Air Monitoring Directive, Chapter 9 Reporting prescribes who is required to submit realtime ambient air monitoring data and the criteria for which parameters are required to be submitted (this is dependent on what is monitored). The Alberta Real-time Ambient Air Website is made accessible to the public at: Data Submission File The file format is proprietary and has been maintained through many iterations of the real-time ambient data system. This means that there are some components in the file formatting that may not be used, nevertheless, must be included in the file as placeholders so that the data will continue to load properly. One file shall be transmitted for each station for each hour of data collected. Multiple stations cannot be combined into a single file, nor can multiple hours be combined into a single file. File Name The file name must be in the following format: H##_YY_MM_DD_STNAM.TXT Example: H13_17_01_03_04PAS.TXT Where: Page 1 of 9

2 ## is the two digit hour in which the data was collected. For single digit hours a leading zero shall be included (example: 8:00 AM will be H08 not H8 ). This is to be measured at the end of the 1 hour collection period and always in Mountain Standard Time (MST) using a 24-hour format. YY is the two digit year in which the data was collected. In single digit years a leading zero shall be included (not applicable for a long time to come). MM is the two digit month in which the data was collected. In single digit months a leading zero shall be included (example: March will be 03 not 3 ). DD is the two digit day in which the data was collected. For single digit days a leading zero shall be included (example: the fifth day shall be 05 not 5 ). STNAM is the Station Abbreviation assigned (usually numbered in order they are created) by the system administrator. These are set when a new station is brought online and generally do not change even when a station is moved. For example, a file called H11_17_02_04_12AQM.TXT would contain hourly data collected from 1000 to 1100 MST on February 4th 2017 for the Edmonton East Station (12AQM). File Format A typical file: IQUA: 17/01/0313:00:00 9PM2 ***** Hourly Readings ***** Station: 11AQM Data Logger: ULT_PLC Date: 17/01/03 YY/MM/DD Time: 13:00:00 MST Parameter,Units,Value,IQUA O3,PPM,0.011,5 NO,PPM,0.0236, NO2,PPM,0.0251,5 NOX,PPM,0.0488, CO,PPM,0.5,1 CH4,PPM,2.3, NMHC,PPM,0.0, THC,PPM,2.3, PM2,UGM,12,9 **************** EOF **************** Line 1: Contains a statement that AQI (formerly IQUA) is being calculated for a certain time. This is a semi-deprecated line but must continue to be included in the files. IQUA: YY/MM/DDHH:MM:SS IIPPP Where: Page 2 of 9

3 YY is the two digit year in which the data was collected. In single digit years a leading zero shall be included (not applicable for a long time to come) MM is the two digit month in which the data was collected. In single digit months a leading zero shall be included (example: March will be 03 not 3 ). DD is the two digit day in which the data was collected. For single digit days a leading zero shall be included (example: the fifth day shall be 05 not 5 ). HH is the two digit hour in which the data was collected. For single digit hours a leading zero shall be included (example: 8:00 AM will be 08 not 8 ). This is to be measured at the end of the 1 hour collection period and always in Mountain Standard Time (MST) using a 24-hour format. MM (second occurrence) is the two digit minute in which the data was collected. For the purposes of the real-time ambient data system this value should always be 00. SS is the two digit second in which the data was collected. For the purposes of the real-time ambient data system this value should always be 00. II contains the calculated IQUA. This is no longer used so any number is acceptable is preferred if the provider is not calculating this value and is also to be used if the value cannot be calculated in a given hour. PPP contains the Parameter Abbreviation (as per Table 1) that is driving the calculated IQUA. This is no longer used. If the data provider chooses not to calculate this value, any Parameter Abbreviation can be used, but O3 is preferred. Line 2: This line is whitespace. Line 3: This line is fixed and contains the text ***** Hourly Readings *****. Line 4: This line is whitespace. Line 5: This line contains station information, telling the loader software which station the file is for. Station: ABBREV The line contains the fixed text Station: followed by a single space and the Station Abbreviation as assigned by the administrator. Line 6: This line contains data logger information (what logger was used to collect the data). Data Logger: LOGGER The line contains the fixed text Data Logger: followed by one space and then the name of the data logger used. This name is determined by the provider. Line 7: This line contains the date of sample collection. Date: yy/mm/dd YY/MM/DD Where: Page 3 of 9

4 This line contains the fixed text Date: followed by one space followed by the variable date of collection in the format below, followed by the fixed text YY/MM/DD. yy is the two digit year in which the data was collected. In single digit years a leading zero shall be included (not applicable for a long time to come) mm is the two digit month in which the data was collected. In single digit months a leading zero shall be included (example: March will be 03 not 3 ). dd is the two digit day in which the data was collected. For single digit days a leading zero shall be included (example: the fifth day shall be 05 not 5 ). Line 8: This line contains the time of sample collection. Time: HH:MM:SS MST Where: This line contains the fixed text Time: followed by one space followed by the variable time of collection in the format below, followed by the fixed text MST HH is the two digit hour in which the data was collected. In single digit hours a leading zero shall be included (example: 8:00 AM will be 08 not 8 ). This is to be measured at the end of the 1 hour collection period and always in Mountain Standard Time (MST) using a 24-hour format. MM is the two digit minute in which the data was collected. For the purposes of the real-time ambient data system this value should always be 00. SS is the two digit second in which the data was collected. For the purposes of the real-time ambient data system this value should always be 00. Line 9: This line is whitespace Line 10: This line denotes the parameter data transmission format. It contains the fixed text Parameter,Units,Value,IQUA. Line 11: This is a fixed line denoting the start of the parameter data. It contains the fixed text Lines 12 End of Data: Each of the next lines contains the data for a single parameter for a single hour in the following format: PPP,UUU,VVVV,II Where : PPP is the Parameter Abbreviation as provided in Table 1. UUU is the Unit Abbreviation as provided in Table 1. Page 4 of 9

5 VVVV is the recorded value of the monitored parameter for the hour indicated on line 8. If no value is recorded for the hour then this shall be n/a. Using or also works but is technically offspec and only works because of manual filters set up by the administrator. II is the parameter specific IQUA as calculated by the provider. This is no longer needed, but a value is still required. If the provider chooses not to calculate IQUA or a specific parameter is not used for IQUA then this value should be Final Line: After all parameter value lines are complete the final line shall denote the end of the file. This line will contain the fixed text ***************EOF***************. Important Notes on File Format End of Line: At the end of a line, the Windows END OF LINE Standard (known as CR LF) must be used. This must be correct at the Data Acquisition System before the file is transferred via SFTP (Secure File Transfer Protocol). Unix or Macintosh standards are not acceptable. Generally if the file looks right in Microsoft Notepad it will have the correct end of line characters. Time: The collection time is always at the end of the hour in which the data was collected and is always in Mountain Standard Time (MST). Hours will be in a 24-hour format with midnight being the first hour in a new day and using the hour code 00. Single digit hours will include a leading zero; 01, 02, 03, etc. Since midnight is considered the first hour of a day and uses the code 00, there will be no hour 24. Instead the last hour of the day will be 11pm which will be coded as 23. Because the time format is based on the end of the data collection period and not the start of the collection period, data collected between 11pm and midnight will be considered data for the day after it was collected. This applies to days, months and years when a new day, new month or new year is starting. For example the data collected from 11pm to midnight on December 31st 2016 would be reported as H00_17_01_01_ABBREV. This applies to both the file name and the internal file format. Data The real-time ambient data system requires hourly data. Since most continuous instrumentation records data in smaller intervals than one hour, this data will have to be converted to hourly averages. The Air Monitoring Directive Chapter 9 Reporting provides requirements for data precision and rounding for submission of ambient air monitoring data to Alberta s Ambient Air Quality Data Warehouse as well as reports submitted to the Director. These principles also apply for the submission of real-time ambient data. Units The real-time ambient data system has fixed units for each parameter. It is not currently capable of converting from one unit to another. For example, if a parameter requires units of parts per million and data is submitted in parts per billion, the system will collect and display data 1000 times higher than it Page 5 of 9

6 actually is. For this reason, there is only one fixed unit for each parameter and all data providers must use the consistent units. The units required for each parameter are listed in Table 1. Parameters The parameters in Table 1 are required to be submitted electronically in real-time when the conditions in AMD Chapter 9 (RC 13-S) are met. Table 1 Required real-time parameters, parameter abbreviations and unit codes Parameter Abbreviation Unit Unit Code Ammonia NH3 Parts per Million PPM Benzene BNZ Parts per Billion PPB Carbon Monoxide CO Parts per Million PPM Ethylbenzene EB Parts per Billion PPB Ethylene C2H4 Parts per Million PPM Fine Particulate Matter PM2 Micrograms per Cubic Metre UGM Hydrogen Sulphide H2S Parts per Million PPM Inhalable Particulate Matter PM10 Micrograms per Cubic Metre UGM Methane CH4 Parts per Million PPM Nitric Oxide NO Parts per Million PPM Nitrogen Dioxide NO2 Parts per Million PPM Non-Methane Hydrocarbons NMHC Parts per Million PPM Outdoor Ambient TPX Degrees Celsius DEGC Temperature Ozone O3 Parts per Million PPM Reactive Oxidized Nitrogen ROY Parts per Million PPM Relative Humidity RH Percent % Styrene STR Parts per Billion PPB Sulphur Dioxide SO2 Parts per Million PPM Toluene TOL Parts per Billion PPB Total Hydrocarbons THC Parts per Million PPM Total Oxides of Nitrogen NOX Parts per Million PPM Total Reduced Sulphur TRS Parts per Million PPM Wind Direction WDR Degrees DEG Wind Speed WSP Kilometres per Hour KPH Xylene XYL Parts per Billion PPB The meteorological parameters listed in Table 2 are also accepted in real-time, but are not required to be reported in real time. If reported, the meteorological parameters must be provided in the units specified in Table 2. Page 6 of 9

7 Table 2 Optional real-time meteorological parameters, parameter abbreviations and unit codes Parameter Abbreviation Unit Unit Code Barometric Pressure* BP Kilopascals KPA Solar Radiation SR Watts per Square Metre W/M2 * Barometric pressure is not sea-level adjusted. If a pollutant parameter is monitored and does not appear in Table 1, the system administrator must be notified so that an abbreviation can be added to the system and to Table 1. If one parameter is collected by two (or more) analyzers/methods at one station, the second (or additional) analyzer is generally not to be included in the real-time data. Exceptions can be made if the dual collection is intended to be long-term or permanent. These must be brought forward to the system administrator. Possible common examples where exceptions may or may not be made: 1. Temperature is collected at multiple heights: if outdoor ambient temperature is collected at ground level and multiple heights (10 metre, 20 metre, etc.) these can be transmitted if desired after obtaining authorization from the system administrator. In general this will require using the parameter code of TPX10, TPX20, etc. 2. Fine particulate matter is collected using two or different methods at one monitoring station: a) If this is for testing purposes and after test completion the analyzer will be removed, then during the testing period, only data from the primary (currently trusted) analyzer shall be transmitted. b) If the additional analyzer is intended to be permanent then one analyzer must be chosen as the primary/trusted analyzer and its data transmitted under the PM2 parameter code. The standard for reporting dual collection would be to append a -2 to the parameter abbreviation for the additional analyzer. The unique parameter code is required because the data loader needs to differentiate between analyzers. In the case of Air Quality Health Index (AQHI) stations, one trusted analyzer must be chosen as the PM2 fine particulate analyzer for the AQHI calculation. E.g., if two fine particulate matter analyzers are reporting for one monitoring station, the primary analyzer would report using the abbreviation PM2 and the secondary would report using PM2-2, after agreement from the system administrator. Data Transmission/Accounts/Server Hourly files for each station shall be sent to an SFTP server located at sftp.gov.ab.ca. Each data provider will be provided with a unique account on the SFTP server. Accounts shall be kept secure and not shared between providers, even where the same employees/contractors are providing their services to multiple data providers. Contact the system administrator to obtain account access or to make changes to accounts. Submission Deadline Real-time ambient data must be received by the SFTP server on or before 24 minutes past the hour in which it was collected. Earlier is preferred as it allows the public-facing websites to inform the public of air quality conditions faster. Page 7 of 9

8 The 24 minute limit is for multiple reasons: 1. To ensure that data is available to the public no more than 30 minutes after the hour in which it was collected. This data is used to calculate the AQHI, which is used to alert the public of changes in air quality and enable them to protect themselves from exposure to air pollution. 2. To allow the processing time necessary to load the data and convert it into the format used by Environment and Climate Change Canada, as their data system loads the data by thirty minutes past the hour to calculate and report the AQHI as well as to provide timely forecasting of air quality. Varying submission times from provider to provider are noticeable on Alberta s Real-time Ambient Air Website as well as other data sources. The real-time ambient data used to calculate the AQHI also feeds the AQHI Canada mobile app as well as Environment and Climate Change Canada s AQHI website, Weather Office and the Weather Network. The web hits for the AQHI data service has been around the 5-6 million mark the past couple years, so the real-time data is well used and scrutinized in the public eye. This is why it is important that the data be up-to-date and available as much as is practicable. Omissions and Errors If there are current issues transmitting real-time ambient data, or if it is known that there will be issues preventing real-time data submission, the data provider shall contact Alberta Environment and Parks (AEP) Air Quality Notifications with any known details on the issues and timeline. This could be very important information should there be an air quality event (such as forest fire smoke) where data is being tracked by the Government of Alberta, Environment and Climate Change Canada, or the public. If erroneous data has been submitted (e.g., elevated values transmitted as a result of instrument malfunction, power outage, data transmission during calibration, etc.), the data provider shall contact AEP Air Quality Notifications to notify that erroneous data has been submitted. This allows the AEP to remove any erroneous data from the real-time data service, and whenever possible to prevent transmission to Environment and Climate Change Canada, the AQHI Canada App, and ultimately the public. This is intended to capture invalid elevated pollutant concentrations over a period 72 hours following original submission. While 24-7 response and notification is not required, the sooner AEP is notified of errors, the sooner errors can be rectified. Notification of erroneous data submitted can be ed to AEP Air Quality Notifications. In some cases, data providers may be contacted by AEP about data omission or potential erroneous data submitted. Under this situation the data provider must confirm whether or not the data in question is erroneous or valid, and, whenever possible, resubmit data which has not been successfully transmitted. If data providers are unsure whether data transmission has been successful (e.g., following power outage, connectivity disruption, or data polling errors), Alberta s Real-time Ambient Air Website may be referred to or data providers can contact the system administrator. AEP Air Quality Notifications can be reached by at: AEP.AQHI@gov.ab.ca Contacting the System Administrator The system administrator can be reached by at: AirQuality.WebMaster@gov.ab.ca Page 8 of 9

9 Original signed by: Date: Hamid Namsechi, Director Air Policy Environment and Parks Page 9 of 9