ETL for Data Center Operation Administration Guide 7EN /2016

Transcription

1 StruxureWare Power Monitoring Expert 8.1 ETL for Data Center Operation Administration Guide 7EN /2016

2 Legal Information Legal Information StruxureWare Power Monitoring Expert The Schneider Electric brand and any registered trademarks of Schneider Electric Industries SAS referred to in this guide are the sole property of Schneider Electric SA and its subsidiaries. They may not be used for any purpose without the owner's permission, given in writing. This guide and its content are protected, within the meaning of the French intellectual property code (Code de la propriété intellectuelle français, referred to hereafter as "the Code"), under the laws of copyright covering texts, drawings and models, as well as by trademark law. You agree not to reproduce, other than for your own personal, noncommercial use as defined in the Code, all or part of this guide on any medium whatsoever without Schneider Electric's permission, given in writing. You also agree not to establish any hypertext links to this guide or its content. Schneider Electric does not grant any right or license for the personal and noncommercial use of the guide or its content, except for a non-exclusive license to consult it on an "as is" basis, at your own risk. All other rights are reserved. Electrical equipment should be installed, operated, serviced and maintained only by qualified personnel. No responsibility is assumed by Schneider Electric for any consequences arising out of the use of this material. As standards, specifications and designs change from time to time, please ask for confirmation of the information given in this publication. Page 2 of Schneider Electric. All rights reserved.

3 StruxureWare Power Monitoring Expert Safety Information Safety Information Important Information Please Note Read these instructions carefully and look at the equipment to become familiar with the device before trying to install, operate, service or maintain it. The following special messages may appear throughout this bulletin or on the equipment to warn of potential hazards or to call attention to information that clarifies or simplifies a procedure. The addition of either symbol to a "Danger" or "Warning" safety label indicates that an electrical hazard exists which will result in personal injury if the instructions are not followed. This is the safety alert symbol. It is used to alert you to potential personal injury hazards. Obey all safety messages that follow this symbol to avoid possible injury or death. DANGER DANGER indicates a hazardous situation which, if not avoided, will result in death or serious injury. WARNING WARNING indicates a hazardous situation which, if not avoided, could result in death or serious injury. CAUTION CAUTION indicates a hazardous situation which, if not avoided, could result in minor or moderate injury. NOTICE NOTICE is used to address practices not related to physical injury. Electrical equipment should be installed, operated, serviced and maintained only by qualified personnel. No responsibility is assumed by Schneider Electric for any consequences arising out of the use of this material. A qualified person is one who has skills and knowledge related to the construction, installation, and operation of electrical equipment and has received safety training to recognize and avoid the hazards involved Schneider Electric. All rights reserved. Page 3 of 24

4 Safety Precautions Safety Precautions StruxureWare Power Monitoring Expert UNINTENDED EQUIPMENT OPERATION WARNING Do not use the software for critical control or protection applications where human or equipment safety relies on the operation of the control action. Do not use the software to control time-critical functions because communication delays can occur between the time a control is initiated and when that action is applied. Do not use the software to control remote equipment without securing it with an authorized access level, and without including a status object to provide feedback about the status of the control operation. Failure to follow these instructions can result in death or serious injury. INACCURATE DATA RESULTS WARNING Do not incorrectly configure the software, as this can lead to incorrect reports and/or data results. Do not base your maintenance or service actions solely on messages and information displayed by the software. Do not rely solely on software messages and reports to determine if the system is functioning correctly or meeting all applicable standards and requirements. Consider the implications of unanticipated transmission delays or failures of communications links. Failure to follow these instructions can result in death, serious injury, or equipment damage. Page 4 of Schneider Electric. All rights reserved.

5 StruxureWare Power Monitoring Expert Contents Contents Legal Information 2 Safety Information 3 Safety Precautions 4 Contents 5 ETL for Data Center Operation (DCO) 7 Installing the ETL for Data Center Operation 7 Supported Measurements 8 Before using ETL for DCO 9 Validating device logging 10 Validating devices are linked to your hierarchy 12 Importing previously saved ETL jobs 13 Create a DCO ETL job 14 Run an ETL job 17 Running an ETL job once 17 Running the ETL job as a service 18 Manage ETL jobs Schneider Electric. All rights reserved. Page 5 of 24

6 Contents StruxureWare Power Monitoring Expert Page 6 of Schneider Electric. All rights reserved.

7 StruxureWare Power Monitoring Expert ETL for Data Center Operation (DCO) ETL for Data Center Operation (DCO) The Extract Transform Load (ETL) Engine is used to extract data from one application, perform a transformation on that data so it can then be loaded into another application. An ETL job consists of the following tasks: Extract Task Reads data from a data source. Transform Task An optional task that converts the extracted data into a form that is compatible with the target data source. Load Task Writes the data to the target data source. ETL for DCO extracts the following source quantity electrical data from Power Monitoring Expert and Power Monitoring Expert Data Center Edition: Energy Current Power Per phase energy Per phase power ETL for DCO then transforms this data into a format that is loaded into a DCO staging database. INACCURATE DATA RESULTS WARNING Do not incorrectly configure the software, as this can lead to incorrect reports and/or data results. Do not base your maintenance or service actions solely on messages and information passed from the ETL Administration Tool. Do not rely solely on software messages and reports created from data passed from the ETL Administration Tool to determine if the system is functioning correctly or meeting all applicable standards and requirements. Consider the implications of unanticipated transmission delays or failures of communications links. Failure to follow these instructions can result in death, serious injury, or equipment damage. Installing the ETL for Data Center Operation Note You must install the ETL Administration Tool using a Windows Administrator account. To install ETL for Data Center Operation (DCO): 1. Download ETL for DCO from the PME Exchange Community Schneider Electric. All rights reserved. Page 7 of 24

8 ETL for Data Center Operation (DCO) StruxureWare Power Monitoring Expert 2. Double-click MainSetup.exe. 3. Application Language: Select your preferred application language from the drop-down list and click Next. Note The ETL Administration Tool supports English only. 4. Welcome screen: Review the steps and click Next. 5. License Agreement: Read the End User License Agreement (EULA) and if you accept the terms of the agreement, click I Agree to proceed. 6. Setup Type: ETL for DCO can only be installed with the Standalone Server option. Click Next. Note To configure ETL for DCO on a distributed system, you must install it using the standalone server setup type. After it is installed edit the extract task's database connection string. See "Installing ETL for Data Center Operation" on page 1 and "Create a DCO ETL job" on page 14 for details. 7. File Destination: Click Next to install ETL to the default location. To select a different location, click the Browse button and select a new location. Click OK. Note If you are upgrading ETL for DCO from a previous version you cannot select the install location. 8. Check System: The installer checks the operating system. If there is an issue, the installer notifies you to correct it. 9. Ready to Configure: The configuration choices are displayed before being installed. Ensure that all items are correct before proceeding. Click Back to move back through the installer and change any items. 10. Click Install to continue. The Copy Files screen appears and the ETL files are copied to the system. 11. Configure System: The selected configuration settings are applied. 12. Complete: The Complete page appears after the install is successful. Click Installation Log to view details recorded for the installation process. 13. Click Close to finish. Tip To verify the ETL for DCO version in Control Panel > Programs and Features click ETL for Data Centers (DCO). The version information appears along the bottom. Supported Measurements You must configure the meter to log the measurements that you want to load into Data Center Operation (DCO). Meters that provide the following power, per phase power, energy, per phase energy, and current measurements can be extracted from Power Monitoring Expert or Power Monitoring Expert Data Center Edition, and then transformed and loaded into DCO: Power: Page 8 of Schneider Electric. All rights reserved.

9 StruxureWare Power Monitoring Expert ETL for Data Center Operation (DCO) CDM Measurement Unit Quantity Name in ION Database Block Demand Active Power kw Active Power Sliding Window Demand Delivered- Received Block Demand Active Power Into The Load kw Active Power Sliding Window Demand Delivered Active Power Mean kw Active Power Mean Active Power kw Active Power Per Phase Power: CDM Measurement Unit Quantity Name in ION Database Active Power A kw Active Power Phase A Active Power A kw Active Power Phase B Active Power A kw Active Power Phase C Energy: CDM Measurement Unit Quantity Name in ION Database Active Energy kwh Active Energy Delivered Received Active Energy into the Load kwh Active Energy Delivered Per Phase Energy: CDM Measurement Unit Quantity Name in ION Database Active Energy into the Load A kwh Active Energy Delivered Phase B Active Energy into the Load B kwh Active Energy Delivered Phase B Active Energy into the Load C kwh Active Energy Delivered Phase B Current: CDM Measurement Unit Quantity Name in ION Database Current Average Mean A Current Phase Average Mean Block Demand Current Avg Into the Load A Current Phase Average Sliding Window Demand Delivered Current Avg A Current Phase Average Note Power, per phase power, energy, and current measurements not listed above can also be supported by this ETL. However, in order to do so you must edit the Power Monitoring Expert or the Power Monitoring Expert Data Center Edition database. See the PME Exchange Community. (Use 'Topic equivalency' as a search term, and then open the discussion addressing adding a custom current quantity.) There is no per phase energy topic equivalency. Before using ETL for DCO Before using the ETL Administration Tool for Data Centers (DCO), you must validate the following: All devices whose measurements you want to load into DCO are logging the supported measurements Schneider Electric. All rights reserved. Page 9 of 24

10 ETL for Data Center Operation (DCO) StruxureWare Power Monitoring Expert The devices whose measurements you want to load into DCO are linked to nodes in your hierarchy. Access to the DCO application, including the staging database, is available. Validating device logging Before you can transfer data from Power Monitoring Expert or Power Monitoring Expert Data Center Edition to DCO, you must first validate that the device is logging measurements that are supported for transfer. To validate device logging: 1. In Management Console, locate the device whose measurements you want to validate. Using a device called PDU_A01.Inc as an example: 2. Open SQL Server Management Studio. 3. Start a new query to find the source ID by running a query similar to the following: SELECT * FROM ION_Data.dbo.Source WHERE DisplayName like '%PDU_A01.I%' Note % symbols are used in the above query to search for all sources containing the text between %. For the PDU_A01.Inc example, the source ID is 8: 4. Use the source ID to run the following query: SELECT DISTINCT Q.ID, Q.Name FROM ION_Data.dbo.DataLog2 DL JOIN ION_Data.dbo.Quantity Q ON DL.QuantityID = Q.ID WHERE DL.SourceID = 8 The query returns the quantity names that are logging to the database. Page 10 of Schneider Electric. All rights reserved.

11 StruxureWare Power Monitoring Expert ETL for Data Center Operation (DCO) 5. Cross reference the quantity names (the Name column values) to the Quantity Name in ION Database column in "Supported Measurements" on page 8. For the PDU_A01.Inc example, the quantity names highlighted below are supported: 2016 Schneider Electric. All rights reserved. Page 11 of 24

12 ETL for Data Center Operation (DCO) StruxureWare Power Monitoring Expert Validating devices are linked to your hierarchy In order for the ETL to extract data from Power Monitoring Expert or Power Monitoring Expert Data Center Edition, your system's devices must be linked to nodes in your hierarchy. To validate that a device is linked to a node in your hierarchy: 1. Open Hierarchy Manager. 2. Click Electrical Equipment. 3. Enter the name of your device as it appears in Management Console in Device <Filter> and press Enter. If the device is mapped to a node in your hierarchy, it will appear in the list. If not, you will need to add it. See "Hierarchies" on page 1for more information. For the PDU_A01.Inc example, the hierarchy node name is Electrical Equipment View Example: Page 12 of Schneider Electric. All rights reserved.

13 StruxureWare Power Monitoring Expert ETL for Data Center Operation (DCO) 4. In the ETL, when you click Recommended Pairs in the Mappings tab, you will see the mappings for your source. For example: Notice that in Source Quantity Pairs > Sources section, the Start Name is the same as the name of the Electrical Equipment hierarchy node (Electrical Equipment View Example). As well, the Source Quantity Pairs > Associated Quantities match the quantity names in the database. Importing previously saved ETL jobs ETL jobs that were saved using a previous version of ETL for DCO can be imported for use in version 3.5. To import previously saved ETL jobs: 1. In Windows Explorer navigate to the previous ETL version. For example:...\program Files\Schneider Electric\ETL 3.0 (DCO) and copy the Jobs folder 2. Backup the Jobs folder to a directory location outside the ETL root folder. 3. Navigate to..\program Files\Schneider Electric\ETL for Data Centers (DCO) and then copy the Jobs folder, overwriting the existing folder. Note The existing ETL jobs will be overwritten. Carefully review any prompts to overwrite files (for example Job.xml), and decide to overwrite or cancel based on your knowledge of the associated jobs Schneider Electric. All rights reserved. Page 13 of 24

14 ETL for Data Center Operation (DCO) StruxureWare Power Monitoring Expert Create a DCO ETL job The following information describes how to create and configure an ETL job for Data Center Operation (DCO). To create a DCO ETL job: 1. Click Start > All Programs > Schneider Electric > ETL Administration Tool (DCO) to open the ETL Administration Tool. Note On Windows Operating Systems with restricted permissions, the ETL tool might not initialize and load its plugins on start up. This is due to limited write permission on the ETL install folder (e.g. C:\Program Files\). The workaround is to install ETL to a custom folder with write permission. 2. Click New to create a new ETL job. 3. Enter the name of the job in the Name field. Important The ETL job name has to be unique. Power Monitoring Expert can use several different ETL release packages; make sure your ETL job name does not conflict with any other ETL jobs on your system. This is particularly important to consider when registering ETL jobs to run as Windows services. 4. Click the Tasks tab to open the Tasks panel. 5. In the Select Category list click Extract. Page 14 of Schneider Electric. All rights reserved.

15 StruxureWare Power Monitoring Expert ETL for Data Center Operation (DCO) 6. In the Select Type list click PowerMonitoringExpertDataCenterHierarchyExtractTask then click Add. The extract task name and description appear under the Extract heading: 7. Configure the extract task settings. See "Configure the extract task" on page 16 for details. 8. In the Select Category list click Transform. 9. In the Select Type list click DcoTransformTask and click Add. The transform task name and description appear under the Transform heading: 10. In the Select Category list click Load. 11. In the Select Type list click DcoLoadTask and click Add. The load task name and description appear under the Load heading: Tip If you want to view the ETL job output in a browser (before loading it in the DCO staging database), you can add an HTMLFileLoadTask load task instead of a DcoLoadTask. To view the HTMLFileLoadTask output, however, you must manually map sources and quantities. See "Mapping configuration" on page 96 for details Schneider Electric. All rights reserved. Page 15 of 24

16 ETL for Data Center Operation (DCO) StruxureWare Power Monitoring Expert 12. Configure the load task settings. See "Configure the load task" on page 17 for details. 13. Click Apply to save without exiting the job, or click OK to save and exit the job. Configure the extract task After the extract task is added to the ETL job, it must be configured. Click the extract task to bring up the list of configurable settings. Click each setting name to change the value. Some settings are configured by typing in a value, while others are configured by selecting an option from a drop-down list. Setting Name Compatibility Use Optimized Measurement Lookup Database Description Optimized Measurement Lookup interacts with the database layer in such a way that may result in an improvement of overall ETL job performance. Setting Parameters / Recommended Values True Modify the string to represent your system configuration settings. Database Connection String Database Connection String - Measurement Lookups Misc Hierarchy View #1 Hierarchy View #2 Limit Quantities to known list? Maximum data queries per pair Query Window Size (in hours) A database connection string A database connection string for PME measurement lookups. Name of the first hierarchy to query for nodes to export. Name of the second hierarchy to query for nodes to export. Set to True to show quantities that are known quantities for Energy, Power, and Current. Set to False to show all quantities. Becomes active only when initial query for a source quantity pair returns no data. When active, the task queries against an earlier time range. If no data is found, the query moves back to an earlier time range up to a maximum number of times specified in this task setting. The size of the query in hours. Used in conjunction with Maximum data queries per pair task setting. For example: server=localhost\ion;database=ion_ Data;UID=My_Database_ Username;pwd=My_User_ Password; Modify the string to represent your system configuration settings. Electrical Equipment Customer True 7 50 Page 16 of Schneider Electric. All rights reserved.

17 StruxureWare Power Monitoring Expert ETL for Data Center Operation (DCO) Configure the load task After the load task is added to the ETL job, you need to configure the DCO staging database connection string. To configure this string, click in the connection string field, or click the Edit icon at the end of the default database connection string, and modify the parameters to match your system. For example: Server= ; Port=5432;UserId=ETL;Password=ETL;Database=DCOStaging; You need to enter the appropriate PostgreSQL database connection string for the target DCO Staging database. Contact your DCO database administrator to retrieve these details. For more information on PostgreSQL connection strings, see postgresql connection strings. Mapping configuration 1. Click the Mappings tab. 2. Click Auto-discover and auto-map pairs. This is the recommended approach for most cases. Choosing this option tells the job to automatically discover all hierarchy nodes with a device attached. This is limited to the two hierarchy views specified in the extract task settings. Alternatively, you can click Use list of sources (allows aliasing) and click Recommended Pairs. Choosing this option returns the sources and quantities available at the time you clicked Recommended Pairs. To discover additional sources and quantities, you must click Recommended Pairs again. Run an ETL job 3. Click OK when finished. Running an ETL job once You can run an ETL job one time or as a service. Running the ETL job as a service uses a default Windows account to run the job repeatedly based on the settings you define in the Job and Tasks panes. You can also run the ETL job as a service using a non-default Windows account. To run the ETL job once: 1. In the Job Management list click the ETL job to run. 2. Click Control. 3. Click Run Once and then click OK in the dialog when it appears. Wait until the output dialog box appears (on very large systems this could take several hours). If the ETL Engine returns a Job execution failed message, click Open Log Folder in the dialog and then open the error log. Locate the timestamp that corresponds to your job and review the log. Based on this information, make the appropriate changes to the job and then run the job again Schneider Electric. All rights reserved. Page 17 of 24

18 ETL for Data Center Operation (DCO) StruxureWare Power Monitoring Expert Running the ETL job as a service You can also register the job to run as a service. Using this method, the job automatically runs at set times based on the settings provided in the Job and Tasks tabs. Before you set a job to run as a service, complete the following tasks: Grant database permissions for the ETL job to run as a service (see below for details). In the Job pane, set the Sleep Time Between Execution value. This value sets the time interval (in seconds) that an ETL job waits before running again. Where applicable: Make sure that the database connections strings are set correctly. Make sure that the ETL job name is unique. Power Monitoring Expert Data Center Edition includes several ETLs; if the ETL job name is being used by another ETL on the system, you will not be able to register it as a service. To install and run the ETL job as a service: 1. In the Job Management list click the ETL job to run. 2. Click Control. 3. Click Register. The ETL Administration Tool creates a registered service under the job name. Click OK in the dialog when it appears. Important If you try to register an ETL job using the name of an ETL job that is already running as a service, you will get a naming conflict error. Rename the ETL job that you want to register; otherwise you might inadvertently unregister the wrong ETL job. 4. Click Start/Continue. To unregister an ETL job service: 1. Click Stop to stop the service, then click Unregister to unregister the service for this job. 2. Click OK to exit the job. Granting database permissions for ETL job run as a service By default, when an ETL job is run as a service it runs under the NT AUTHORITY\SYSTEM Windows user account. With SQL Server 2012 and later, the NT AUTHORITY user does not have adequate database permissions. If an ETL job is run using the NT AUTHORITY user, the ETL cannot connect to the Power Monitoring Expert database and the ETL job fails. For the ETL job to succeed, you may choose to grant database permissions to this user, or see the following Note for other options. To grant database permissions to the NT AUTHORITY user, log in to SQL Server Management Studio as an administrator and run the following script: USE [ION_Data] GO CREATE USER [NT AUTHORITY\SYSTEM] FOR LOGIN [NT AUTHORITY\SYSTEM] GO EXEC sp_addrolemember N'db_owner', N'NT AUTHORITY\SYSTEM' GO USE [ION_Network] Page 18 of Schneider Electric. All rights reserved.

19 StruxureWare Power Monitoring Expert ETL for Data Center Operation (DCO) GO CREATE USER [NT AUTHORITY\SYSTEM] FOR LOGIN [NT AUTHORITY\SYSTEM] GO EXEC sp_addrolemember N'db_owner', N'NT AUTHORITY\SYSTEM' GO Note If security concerns limit you from using the default NT AUTHORITY user, create a dedicated Windows user to run the ETL job as a service: a. Create a Windows user. Note that if the ETL is installed to its default location C:\Program Files\... the Windows user must have Administrator access. b. Set the ETL job to run as a service under the new Windows user. c. Log in to SQL Server Management Studio as an administrator and run the above script, substituting NT AUTHORITY\SYSTEM with the new Windows user. To set the job to run as a service using a non-default account: 1. Open Windows Services. 2. Right-click ETL Engine Service <Job name> and then click Properties. 3. Click This account and enter the account credentials. 4. Click OK. Manage ETL jobs You can set up logging to help manage ETL jobs. You can also switch between ETL jobs, change the order of ETL tasks, and remove ETL tasks from an ETL job. Logging set up Logging lets you enable the various logs where ETL writes the information regarding the status of your ETL job. These logs can assist in tracking down the cause of an unsuccessful ETL job. To enable the ETL logs: 1. Open the ETL Administration Tool. 2. In the Job Management list click the applicable ETL job and then click Edit. 3. Click the Logging tab. The Logging panel appears. 4. For Trace Log, Error Log, and Customer Log, click Enabled as required. 5. (Optional) Provide the location for the log file in the Log File field, or leave at the default location. The default location is: [path]>\program Files\Schneider Electric\<ETL Administration Tool>\Log 6. (Optional) Set the Maximum Log File Size and Maximum Log Files for each log, or leave at the default settings. 7. (Optional) Select the Enabled check box for Notifications and complete the fields for: To Address, From Address, and SMTP Server Address. 8. Click OK when finished to exit the job Schneider Electric. All rights reserved. Page 19 of 24

20 ETL for Data Center Operation (DCO) StruxureWare Power Monitoring Expert Tip After you run an ETL job, the Job Execution Complete dialog appears. You can click Open Log Folder to review the log files. For example: Confirming the ETL job If the ETL Administration Tool returns a Job execution failed message, click Open Log Folder to open the error log. Locate the timestamp that corresponds to your job and review the log. Based on this information, make the appropriate changes to the job and run the job again. Navigate to the location set in the Primary Drop Box setting to confirm the CSV files are being generated and sent to the correct location. Note If you encounter the message 53300: Too many connections for role, contact your DCO staging database administrator to determine the connection limit. The connection limit can change from one DCO installation to another. In cases of tightly constrained connection limits, you may be able to limit the number of connections used at any given time by reducing the Thread Pool Size value of your job. The Thread Pool Size is configurable on the Advanced pane when editing a job. Cloning an ETL job When creating a new job in ETL, you can clone an existing ETL job. To clone an ETL job: 1. In the Job Management list click an existing ETL job. 2. Click Edit. 3. In the Job panel, change the name to define the new ETL job. 4. Click the Task tab and then edit the new ETL job as necessary. Page 20 of Schneider Electric. All rights reserved.

21 StruxureWare Power Monitoring Expert ETL for Data Center Operation (DCO) 5. Click Apply or OK. The ETL job saves with the new name. Sources and quantities are carried over from the original ETL job. It is recommended that you clear the mappings from the cloned ETL job. Renaming an ETL job You can rename an existing job. 1. In the Job Management list click an existing ETL job. 2. Click Edit. 3. In the Job panel, change the name to define the new ETL job. 4. Click OK. 5. Optional: In the Job Management list click the original ETL job and then click Delete. Removing a task from an ETL job 1. In the Job Management list click the applicable ETL job and then click Edit. 2. Click the Tasks tab. 3. Highlight the task that you want to remove from the left pane. 4. Click Delete. 5. Click OK to save and exit job. Switching between ETL jobs To switch between ETL jobs: 1. Click OK at the bottom right to save and exit the current job. 2. In the Job Management list select an ETL job and click Edit Schneider Electric. All rights reserved. Page 21 of 24

22 ETL for Data Center Operation (DCO) StruxureWare Power Monitoring Expert Page 22 of Schneider Electric. All rights reserved.

23

24 StruxureWare Power Monitoring Expert ETL for Data Center Operation Administration Guide Schneider Electric 35 rue Joseph Monier Rueil Malmaison - France ION, ION Enterprise, Modbus, PowerLogic, StruxureWare and Schneider Electric are either trademarks or registered trademarks of Schneider Electric in France, the USA and other countries. Other trademarks used are the property of their respective owners. Electrical equipment should be installed, operated, serviced and maintained only by qualified personnel. No responsibility is assumed by Schneider Electric for any consequences arising out of the use of this material. 7EN Schneider Electric. All Rights Reserved.