Oracle Enterprise Manager Cloud Control 12c: Best Practices for Middleware Management. Mary Peek, Senior Principal Curriculum Developer

Transcription

1 Slide 1 Oracle Enterprise Manager Cloud Control 12c: Best Practices for Middleware Management Mary Peek, Senior Principal Curriculum Developer Hello, and welcome to this online, self-paced course entitled Oracle Enterprise Manager Cloud Control 12c: Best Practices for Middleware Management. My name is Mary Peek, and I ll be your guide for approximately the next two and a half hours of interactive lectures, videos, review sessions, and optional demonstrations. The goal of this course is to teach you best practices when using Oracle Enterprise Manager Cloud Control 12c for managing your WebLogic and SOA, applications and infrastructure.

2 Slide 2 Using the Player Narration and Best Practices Summary Course Outline Player Controls Change View Before we begin, take a look at some of the features of this Flash-based course player. If you've attended a similar self-paced course in the past, then feel free to skip this slide. Let's start with the Outline. It's set up so that you can go at your own pace. Feel free to skip over topics you already feel confident about, or jump right to a topic that really interests you, or go back and review topics that were already covered. Just click a slide title in the outline to display its contents. By default we'll automatically walk you through the slides without requiring you to use the outline. Use the Transcript tab to view the audio transcript for each slide, and use the Search tab to find specific information in the course. Next, let's look at the Player Controls. Use these controls to Pause, Play, or move to the Previous or Next slides. You can also use the interactive Progress Bar to fast forward or rewind the current slide. Some interactive slides in this course may contain additional navigation and controls. This icon on the bottom right corner allows you to change the View of this player, for example to go to Full Screen. You can control the view at any time using this button. A downloadable, audio narration document for this course as well as the Best Practices Summary are available from the Attachments button. This course will now pause, so feel free to take some time and explore the interface. Then when you're ready to continue, click the Next button below or alternatively click About This Course in the outline on the left.

3 Slide 3 PROPERTIES Allow user to leave interaction: Show Next Slide Button: Completion Button Label: Anytime Show always Next Slide Welcome! So, you know the title of the course, but you may be asking yourself, Am I in the right place? To help you answer this question, you can access information here regarding the course objectives, the target audience, and the prerequisites. When finished, click the Next Slide button. What skills will I learn? What can you expect to get out of this course? The core learning objectives are listed here. After this course you should be able to list the most important tasks for successful management of WebLogic Server and SOA applications and infrastructure using Oracle Enterprise Manager Cloud Control 12c. You should also know when to perform each task in the middleware development lifecycle. You should be able to outline why certain management tasks are more important than others. And you should be able to provide examples of the most important management tasks. Who is the target audience? This course is intended for System Administrators and SOA Architects. What are the prerequisites? Before taking this course, you should have experience using Oracle Enterprise Manager Cloud Control 12c as well as the Fusion Middleware product stack. In Particular Oracle WebLogic Server and Oracle SOA Suite. From now on I ll refer to Oracle Enterprise Manager Cloud Control as just Cloud Control.

4 Slide 4 Why Take This Course? What s in it for me? What challenges do I face on the job? How can this help me do my job better? So, why take this course? Start by asking yourself, What s in it for me? You will learn Oracle recommended best practices for managing your Fusion Middleware applications and infrastructure using Cloud Control. What challenges do you face on the job? Enterprise middleware applications are becoming increasingly powerful and are also more challenging to manage. How do you keep track of all your WebLogic Server configurations throughout the enterprise? How do you get visibility into your Java EE applications or your SOA composites? How do you track transactions across multiple tiers? This course will answer all those questions and more. And, finally, how can this course help you do your job better? This course shows you what tasks to prioritize in terms of middleware management so you will know what to do at each stage of the enterprise application development process. Knowing these management best practices will help your business achieve peak performance and avoid costly downtime.

5 Slide 5 Road Map Plan Improve Processes Set Operational Goals Introducing Middleware Management Track & Control Changes Monitor Proactively Diagnose Problems Let's look at how this course is structured. First, I'll give you an introduction to middleware management at a high level we'll look at the challenges you face in managing middleware applications and infrastructure, I'll introduce Cloud Control, and I'll take you through the middleware development lifecycle which is what you see here. Then each section of the course will focus in on each stage in the lifecycle in detail.

6 Slide 6 Challenges in Middleware Management Monitoring multiple domains across the enterprise Monitoring applications across tiers Tracking and maintaining configurations Managing capacity to changing workload Managing multi-tier transaction flow Obtaining visibility into SOA services Today s IT organizations are increasingly adopting Java EE, SOA, composite application, and cloud computing that enable them to quickly connect disparate applications and fulfill ever changing business needs. Although these applications offer unprecedented flexibility and agility, they re more challenging to manage. Particularly, how do you centrally monitor multiple WebLogic domains across the enterprise? And how do you monitor your enterprise applications when they are deployed across several tiers and containers? And how do you track and maintain the configurations of numerous servers and applications? And how do you manage capacity to changing, typically increasing, workload patterns? And how do you get visibility into, and manage transaction flow that spans shared components and services and is deployed across several tiers? And how do you get visibility in to the performance of SOA and Oracle Service Bus services? And so on Let s hear from two Product Managers at Oracle on these challenges, first for WebLogic Server management and then for SOA management. Glen Hawkins, Director, Product Management for WebLogic Server discusses the challenges in WebLogic Server management. James Kao, Director, Product Management for SOA discusses the challenges in SOA management.

7 Slide 9 Introducing Enterprise Manager Cloud Control Application performance management JVM diagnostics Configuration & lifecycle management Business transaction management Performance & diagnostics Configuration & lifecycle management Business transaction management All of these middleware management challenges can lead to more business downtime, higher IT costs, and less agility. Oracle Enterprise Manager Cloud Control helps you meet these challenges. Cloud Control is Oracle's single, integrated solution for managing all aspects of the Oracle Cloud and traditional IT environments and the applications running on them. It couples a potent, top-down monitoring approach with a cost-effective automated configuration management, provisioning, and administrative solution. This powerful combination lets you to stay focused on business priorities, while reducing the effort and cost of managing sophisticated applications built on Oracle WebLogic Server and Oracle Fusion Middleware. Oracle offers several management packs that enhance the capabilities of Cloud Control for specific purposes. In this course we'll be looking at two specific management packs the WebLogic Server Management Pack Enterprise Edition and the SOA Management Pack Enterprise Edition. The WLS Management pack provides application performance management, JVM diagnostics, configuration and lifecycle management, business transaction management, and more for Oracle WebLogic Server application environments. The SOA Management pack provides, performance and diagnostics, configuration and lifecycle management, and business transaction management, for a SOA-based environment. So, Cloud Control, licensed with these two packs, gives you visibility across all your middleware infrastructure and applications so you can manage them more effectively. It also reduces the cost and complexity of middleware management, thereby increasing your ROI on middleware investments.

8 Slide 10 PROPERTIES Allow user to leave interaction: Show Next Slide Button: Completion Button Label: Anytime Show always Next Slide Here you can see the Middleware Development Lifecycle. Different aspects of middleware management are used at each stage of this lifecycle. This course will cover the most important middleware management best practices at each stage. Click a label for more information. Plan In the planning stage you need to look at what your middleware management requirements are in terms of what you are going to monitor and what operational procedures your organization has. Here I ll give you best practices on what parts of Cloud Control you need to install and where you need to install them to meet these requirements. Set Operational Goals Before you go into production you need to set your operational goals. Here I ll give you best practices on defining your monitoring standards. Monitor Proactively Once you are in production you need to ensure that your Oracle Fusion Middleware and WebLogic deployments are up and performing as expected. Here I ll give you best practices on monitoring your targets proactively so you can detect potential problems before they become severe. Diagnose Problems You need to diagnose problems at all stages in the development lifecycle through Development, Test, QA and production but it is vitally important in production. When your critical business applications go down or are not performing well, your business suffers. Here I ll give you best practices on diagnosing problems quickly and efficiently. Track & Control Changes Tracking configurations is probably one of the most time-consuming and difficult tasks you as an administrator face on a daily basis. Do you know how each of your managed targets is configured and whether they comply with your corporate or regulatory standards? Here I ll give you best practices on tracking and controlling configuration changes.

9 Improve Processes You probably spend a large amount of time performing various lifecycle management operations - such as installing, patching, and configuring WebLogic servers, as well as deploying Java applications and SOA composites, and so on. Here I ll give you best practices on improving your processes and automating common lifecycle management operations.

10 Slide 11 Road Map Plan Improve Processes Set Operational Goals Track & Control Changes Monitor Proactively Diagnose Problems In the planning stage, before you even install Cloud Control, you need to look at what your Middleware management requirements are in terms of what are you going to monitor and what operational procedures your organization has. Here I ll give you best practices on what parts of Cloud Control you need to install and where you need to install them to meet these requirements.

11 Slide 12 Planning Best Practices Deploy Cloud Control universally Expand with business-driven needs So what should you plan for in terms of installing Cloud Control to meet your middleware management requirements? First you should deploy Cloud Control universally for plug-and-play core capabilities with Management Agents on each of your monitored hosts. Then you expand with your businessdriven needs. You should deploy Application, Dependency & Performance, or ADP, and JVM Diagnostics where you need deep diagnostic visibility into your applications. And you should deploy Business Transaction Management where you need visibility into your business processes.

12 Slide 13 Monitor & Manage Everything With One Console User User Load Balancer DB DB DB Cloud Control Console Your current and planned environment will no doubt comprise many hosts running multiple WebLogic Servers, the Java EE, SOA, web applications, and business processes deployed on these servers, Oracle databases, and so on. Here you see an example production environment with multiple databases on Exadata machines, WebLogic Servers on Exalogic machines, some with a mixture of SOA and Java EE applications, Web services, Oracle Service Bus, and business processes deployed to them. A load balancer is also used to improve performance. You can, of course, use the individual product consoles to monitor the status of each of these targets, but it becomes cumbersome to shuttle between multiple console windows and track the performance of each of these targets using so many windows. Cloud Control offers a solution that allows you to monitor and manage the complete Oracle IT infrastructure from a single console. Cloud Control gives you the core infrastructure and component management as well as the metrics, configuration, lifecycle, and operational infrastructure.

13 Slide 14 Deploy Cloud Control Universally User User Load Balancer DB DB OMS OMS Management Repository DB So what should you plan for in terms of installing Cloud Control to meet your middleware management requirements? First, deploy Cloud Control universally for plug-and-play core capabilities then expand with business-driven needs. The Oracle Management Service, or OMS, collects and stores data and renders it on the Cloud Control Console. In large production environments, as you see here, deploy multiple OMSs on WebLogic servers to spread the load and improve efficiency of data flow so you can meet your high availability requirements. Then you can use OMS to push Oracle Management Agents to all hosts that contain targets you want monitored. Agents can be pushed in a bulk manner and don't require manual effort for each install. An Agent is responsible for monitoring all services and components on its host. Core monitoring needs to be deployed and available everywhere so you can use it without worrying about scaling or overhead. The Agents send their monitoring data to the OMS which stores the collected information in the single management repository, a database, for future reference and analysis. When you install Cloud Control, a set of mandatory Oracle Management plug-ins are also deployed by default. These plug-ins include special management capabilities customized to suit Fusion Middleware target types and work in conjunction with the OMS and the Management Agents to monitor every Middleware target in your environment. The WebLogic Server Enterprise Edition and SOA Management Enterprise Edition Management Packs are also installed by default when installing Cloud Control.

14 Slide 15 Expand with ADP and JVM Diagnostics User User ADP ADP ADP JVMD JVMD JVMD Load Balancer Deploy ADP for deep visibility into SOA, Oracle Service Bus, Portal & ADP applications. DB DB OMS JVMD OMS JVMD ADP Management Repository DB ADP Deploy JVMD for deep diagnostics into Java applications. Once you have your Cloud Control infrastructure in place, expand it with your business-driven needs. Cloud Control also comes with some other managers and agents that are critical for middleware management. Application Dependency and Performance, or ADP, analyzes SOA, Oracle Service Bus, Portal, and ADF applications. ADP collects performance measurements, tracks contextual relationships, and summarizes data in real-time while introducing as little overhead as possible. So with ADP, you can understand the complex relationships among the business functions, associated interconnected components, and the underlying runtime environments. Deploy the ADP Manager to a managed server in the Cloud Control domain. Or multiple domains, as you see here, for load balancing and high availability. Then deploy the ADP agents on the production WebLogic servers wherever you need deeper visibility into SOA, OSB, Portal, and ADF applications. Another critical functionality in Cloud Control is JVM Diagnostics, or JVMD, which allows you to diagnose problems in Java applications in your production environment. It provides deep diagnostics into Java applications - to the Java thread level and can even provide line-of-code visibility. JVM Diagnostics can help resolve issues with applications related to java locking, memory leaks, hung threads, and so on. And it does this with near-zero overhead, making it ideal to be always-on in production to catch intermittent issues that are extremely hard to reproduce and diagnose. Deploy the JVMD Manager to a managed server in the Cloud Control domain. Or multiple domains, as you see here, for load balancing and high availability. Then deploy the JVMD agents on the production WebLogic servers wherever you need deeper diagnostic visibility into Java applications. That could include systems that are also involved in SOA, Oracle Service Bus, Portal or ADF because those are also Java based, if you need that visibility. Note, you should deploy each of the Diagnostics Managers to their own managed server with no other applications running on those servers.

15 So Oracle does not recommend deploying both ADP and JVMD agents universally. Only deploy them where you need that deep diagnostic visibility into your applications.

16 Slide 16 Expand with Business Transaction Management User User ADP BTM ADP BTM JVMD ADP BTM JVMD JVMD Load Balancer Deploy BTM for real-time monitoring and instance tracking of transactions. BTM DB DB OMS JVMD OMS JVMD BTM ADP ADP DB Management Repository DB If you need visibility into your business transactions, further expand your Cloud Control infrastructure with Business Transaction Management, or BTM. This visibility is an important piece of middleware management as these transactions span multiple tiers and applications. BTM provides real-time monitoring and instance tracking of cross tier and cross application transactions. With it, you can non-invasively view transaction performance and conditions or faults centered on transaction performance. You can also perform payload capture, search and correlation. BTM gives you understanding of your logical transaction flow. Both the WebLogic Server and the SOA Management Packs come with BTM. In the case of the WebLogic Server Management Pack, BTM monitors Java EE services running on WebLogic Server. In the case of the SOA Management Pack, BTM monitors SOA services running on the SOA Suite. So they do not overlap. Note, however, that BTM is a separate install. The Central BTM Servers manage the BTM environment and contain the service-level and transaction management components. Deploy these three BTM server applications to a WebLogic server. They also require an Oracle database for storing performance data, logging messages, and so on. BTM Observers monitor messages and calls between the components of your applications. Decide which business transactions need to be managed then deploy BTM Observers onto the WebLogic Servers involved in those transactions. BTM Monitors collect application performance and usage measurements from observers then pass this data onto the Central BTM servers. Deploy the monitor application to a WebLogic Server.

17 Slide 17 Real User Experience Insight Install RUEI to monitor and manage your end user experience. Another management tool that you may want to install is Real User Experience Insight, or RUEI. Install RUEI to proactively monitor and manage your end user experience. You can see who your users are, where are they coming from, what they re buying or not buying, what kind of user experience they are getting, whether they re running into errors, etc. RUEI collects, processes and presents every detail of your end user experience. This is a separate product from Cloud Control but there is tight integration between the two so you can drill down directly from RUEI into Cloud Control to view the deeper monitoring and diagnostics data or into BTM to view transaction data.

18 Slide 19 Planning Summary Deploy Cloud Control universally Expand with business-driven needs by selectively deploying: ADP for SOA, OSB, Portal, and ADF applications JVMD for Java applications BTM for transactions RUEI for user experience management Now let's summarize the best practices we covered in this planning stage of the development lifecycle. First deploy Cloud Control universally and then expand with your business-driven needs. Deploy ADP where you need deep visibility into SOA, OSB, Portal, and ADF applications. Deploy JVMD where you need deep diagnostic visibility into Java applications. Deploy BTM where you need visibility into your transactions. And install RUEI to monitor and manage your user experience.

19 Slide 20 Road Map Plan Improve Processes Set Operational Goals Track & Control Changes Monitor Proactively Diagnose Problems Before you go into production, you want to set your operational goals. This is where you decide what to monitor and when you want to be alerted if a target is not performing as expected. Here I ll give you best practices on defining your monitoring standards so you can effectively monitor the whole range of middleware targets in your enterprise.

20 Slide 21 Setting Operational Goals Best Practices Define Metric Thresholds Create Monitoring Templates Create Template Collections Create Administration Groups First you should define your metric thresholds so you can be proactively alerted of problems before they occur. Next you should create monitoring templates to group together thresholds by target type and simplify managing and monitoring many targets. Then you should group together monitoring templates of different types to create template collections. Also you should create Administration groups to group together different target types. This is what you associate the monitoring template collections with in the next stage to further ease managing and monitoring a large number of components in your enterprise. These are the most important tasks at this stage. Let s look at each of these in turn.

21 Slide 22 What are Metric Thresholds? Threshold crossed Alert / Page Administrator Metric thresholds are very important for any type of management and monitoring and, in fact, are often called the "bread and butter of monitoring". So defining these is an extremely important task. You can specify metric thresholds for each performance metric that Cloud Control monitors. A metric threshold causes an alert to be triggered when a particular performance metric value exceeds the limit. For example, when the response time on a critical application, becomes very slow. Often you're not aware of performance problems until users call to complain. Using rules and notifications, which we will discuss later, you can be sent an and/or be paged when a threshold is crossed. This enables you to be more proactive in monitoring the availability and performance of your Middleware and the applications it supports. So thresholds let you detect impending IT problems. At this stage, you define the operational goals, that is, the thresholds, and then in the next stage, as you go into production, you'll set up what to do with the alerts by creating rules and notifications.

22 Slide 23 Metric Threshold Example JVM Heap Usage % Critical Alert Warning Alert 1 PM 2 PM 3 PM 4 PM You can set metric threshold values for two levels of alert severity warning or critical. For example, you want to monitor the JMV Heap Usage on a critical WebLogic Server. So you define a warning threshold of greater than 80%. Warning means that attention is required but the area is still functional. And you define a critical threshold of greater than 90%. Critical means immediate action is required because the area is either non-functional or there is an imminent problem. When the JVM heap usage exceeds 80% a warning alert is created and you should take corrective action. When the JVM heap usage exceeds 90% another alert is created and you need to take immediate action. Development, Test and Production systems will have different thresholds. In Development and Test you want the thresholds to be achievable so you set them based on the data you have in those environments. But you don't want to wait until you go into Production to define your thresholds. You need to set them up in Test, based on test data, then refine them as you go into production.

23 Slide 24 PROPERTIES Allow user to leave interaction: Show Next Slide Button: Completion Button Label: Anytime Show upon completion Next Slide So how do you decide what to set metric thresholds on and what to set them to? Follow this 5- step procedure to define your metric thresholds. Click through the numbered steps at the bottom to proceed. Determine Critical Components To decide what components to set your thresholds on, determine your critical components involved in delivering your services. These are your WebLogic Servers, your Java EE applications, SOA composites, and so on which are all target types in Cloud Control. Choose Metrics Look at the metrics available in Cloud Control for each of the target types for your critical components and decide which are the important metrics that you need to be alerted on if they go out of normal range. For example, you may want to be alerted if your WebLogic Server Datasource Failed Waiting Connection Requests exceeds a certain limit. And you probably want to know if the average response time on a critical SOA composite becomes too long. Choose only the metrics you care about. You should only set thresholds for metrics whose events you will manage. You do not want to generate unnecessary metric alerts. You may also want to disable metric collection for those metrics you do not need to avoid unnecessary collection and storage of those metric values. Determine Acceptable Performance To determine what to set the threshold values to, analyze your environment during load testing and determine when performance is acceptable for a typical workload. That is when it meets your service level agreements. Define Thresholds While performance is acceptable, you can identify the threshold values for key performance indicators by viewing performance metrics in Cloud Control. Here you re looking at your critical business applications. You decide on the service level you require and determine your thresholds from that. When defining thresholds, the key is to choose acceptable values to avoid

24 unnecessary alerts, while still being notified of issues in a timely manner. Be conservative with critical thresholds reserve these for high signals of a serious problems! Note that a few metric thresholds come predefined out of the box. Although these values are acceptable for most monitoring conditions, your environment may require customized threshold values to reflect the operational norms of your environment accurately. Fine-Tune Once thresholds are set, you can then fine-tune these values over time until your system meets or exceeds your performance goals. And Cloud Control can also make recommendations for thresholds based on historical data, however, this would only apply once you ve been in production for a while.

25 Slide 25 Create Monitoring Templates Monitoring Production Template SOA Composites Threshold Avg Response 1 Time > 10/20 Threshold Business Faults 2 > 10/15 Threshold Errors > 5/10 3 Production Servers To actually set your thresholds, you create monitoring templates. These allow you to set one or more threshold values for a particular target type. Then later you can apply them to one or more targets of that type. For example as you see here, you want to set performance thresholds on all your production SOA composite applications. You decide that Average Response Time, Business Faults and Errors are what you want to monitor and you define the thresholds for each of these. Then you create a monitoring template called Production SOA Composites and actually set the threshold values. Then you can apply these thresholds to all your production SOA Composite applications running on your production servers. This simplifies the task of standardizing monitoring settings across your enterprise because you set specific thresholds once and then apply the template to the appropriate WebLogic Servers, applications, and services. You want to avoid setting thresholds individually, server by server or application by application, instead create templates that reflect your overall business Service Level Agreements (or SLAs) and then mass apply them to your whole environment. (Here we re talking about simple business SLAs that translate into thresholds in Cloud Control, for example, you want response time to be less than a minute. There are also more complex SLAs that are used for service provider contracts. These map to the SLA feature in Cloud Control which is beyond the scope of this course so I won t be going into those.) Note that you can create thresholds at different levels of granularity. Generally you want to create broadly applicable thresholds that can be templatized. The templates are the base from which additional fine-grained thresholds may be set. You may need to add in fine-grained thresholds as you encounter specific issues, but typically this will be rare. Also Cloud Control does include predefined templates for monitoring Oracle target types, such as Oracle WebLogic Domain, Oracle Fusion Middleware Farm, Application Deployment, Oracle WebLogic Server, etc. You can use predefined templates as a starting point or create a new monitoring template.

26 At this setting operational goals stage, you create the monitoring templates and you apply them in the next stage as you go into production.

27 Slide 26 Create Template Collections Monitoring Templates Production SOA Composite Production Application Production Threshold 1 Deployment WebLogic Threshold 1 Server Threshold 2 Threshold 3 Threshold 2 Threshold 3 Threshold 1 Threshold 2 Threshold 3 Template Collection Production Template Collection Taking it a another step, you can further ease the administration of large enterprise environments by creating template collections. A template collection is a group of settings used to monitor and manage different types of targets within Cloud Control. A template collection contains a collection of different monitoring templates. You can only have one monitoring template per target type in the template collection. So for example, as you see here we have one for SOA Composites, one for Application Deployments, and one for WebLogic Servers. You could have others as well. And in this example the template collection contains the settings for production targets. Template collections can also contain compliance standards and cloud policies. But they are beyond the scope of this course and so I won t be going into those here.

28 Slide 27 Create Administration Groups Administration Group Template Collection Production Group Production Template Collection Associate The reason you create template collections is so you can associate them with Administration Groups. Today s IT operations can be responsible for managing a great number of components, such as WebLogic domains, SOA infrastructure, databases, hosts, or other components, which can be time consuming and impossible to manage individually. You can combine targets into logical sets, called Administration groups and then collectively apply monitoring and management settings to those groups. This enables you to organize, manage, and effectively monitor the potentially large number of targets in your enterprise. So you should put together the targets that are monitored and managed in the same way into a group for example all your Production systems could be monitored and managed in one way, versus your non-production systems. Or you could group together all the targets that support the same application. Then you associate a template collection with an Administration Group so all the monitoring settings in the template collection are applied to the members of the group. Then, when a new target is added to the group, Cloud Control automatically propagates monitoring and other management settings using the Template Collection. This completely eliminates the need for administrator intervention. This level of automation simplifies the target setup process and also enables a datacenter to easily scale as new targets are added to Cloud Control. (Administrator privileges also propagate in Administration groups when new targets are added.) At this setting operational goals stage, create the template collections and, in the next stage as you go into production, you associate them with Administration groups. Oracle recommends using template collections to apply your thresholds. Applying monitoring templates directly to targets is recommended only for ad hoc operations.

29 Slide 29 Setting Operational Goals Summary Define metric thresholds Create Monitoring Templates Create Template Collections Create Administration Groups Now let s summarize the best practices we covered in this setting operational goals stage of the development lifecycle. Define metric thresholds on your critical components based on performance that meets your Service Level Agreements. Create monitoring templates to group together thresholds for targets of the same type. Create template collections to group together thresholds for groups of targets of different types. Create Administration Groups to monitor and manage different types of targets in the same way.

30 Slide 30 Road Map Plan Improve Processes Set Operational Goals Track & Control Changes Monitor Proactively Diagnose Problems An integral part of your daily administrator duties is to ensure that your Fusion Middleware and WebLogic deployments are up and performing as expected. So once you are in production it is highly advantageous to proactively monitor your targets for potential problems before they become severe. Here I ll give you best practices on monitoring your targets proactively.

31 Slide 31 Monitoring Proactively Best Practices Apply monitoring thresholds to your targets Set up notifications MANAGE BY EXCEPTION Create reports Customize and build monitoring dashboards Monitor diagnostic findings Monitor transactions with BTM Oracle s recommended general best practice here is to manage by exception. You don t have time to sit all day staring at the Cloud Control console, especially since a lot of the data is quite low level. That s not an effective use of your time. We ll talk more about diagnostics and when to view low-level metrics later but for now you should set up proactive monitoring so you can be notified of impending problems. You ve defined your thresholds for key middleware performance metrics in the Setting Operational Goals stage by creating monitoring templates and template collections, now you need to apply those thresholds to your targets. Then you should set up notifications so you ll be alerted if any of your thresholds are violated. You do that by doing some initial set up then creating rules and rule sets. These are the two most important tasks at this stage. But you should also create reports so you can confirm that your metrics are good. And you should customize and build specific monitoring dashboards in Cloud Control. As I said earlier, it s not an effective use of your time to spend all day looking at the Cloud Control Console but it s a good idea to have a few key dashboards customized to your particular monitoring needs that you can monitor routinely. Also you want to check your important targets for diagnostic findings and view these to be alerted of potential issues. Lastly, to monitor your transactions, use Business Transaction Management or BTM.

32 Slide 32 Apply Thresholds to Targets Define thresholds Apply thresholds to targets Applying monitoring template to targets directly Associating template collection to administration group In the previous stage, you defined your monitoring thresholds. You did this by creating monitoring templates and adding monitoring templates to template collections to make management and monitoring in large environments easier. Now you actually apply those thresholds to your targets by either directly applying monitoring templates to targets or associating template collections with Administration groups.

33 Slide 33 Apply Monitoring Templates Deployed Applications SOA Composites WebLogic Servers Thresholds Monitoring Templates Targets So you've defined all your thresholds of the same target type into monitoring templates. One way you can set your thresholds is by applying these monitoring templates directly to multiple targets of the same type. Here you see how a deployed applications monitoring template can be applied to multiple deployed applications, a SOA composites monitoring template can be applied to multiple SOA composites, and a WebLogic Server monitoring template can be applied to multiple WebLogic Servers. However, Oracle recommends this only for ad hoc operations.

34 Slide 34 Associate Template Collections with Groups Deployed Applications SOA Composites WebLogic Servers Thresholds Monitoring Templates Template Collection Administration Group As a best practice, you should use template collections to set your thresholds. By this stage, you've grouped your monitoring templates into template collections. Each of these template collections contains only one type of each monitoring template. For example as you see here, the template collection contains one deployed application template, one SOA composite template, and one WebLogic Server template. Now you associate a template collection with an Administration group to apply all the thresholds contained in the template collection to actual targets. You'll recall that administration groups are a way to group together targets of different types that are managed and monitored in the same way, for example, all your production systems and applications as you see here. By associating a template collection to an Administration group you collectively apply all the monitoring and management settings to the group in one step rather than having to apply them to each target individually. Then, when a new target is added to the group, Cloud Control automatically propagates the monitoring and management settings using the Template Collection. This allows you to manage many targets as one and is scalable as your enterprise grows. As new targets are added there is now minimal effort to monitor those new targets. So Oracle recommends using template collections and Administration groups as the preferred way to apply your thresholds across your enterprise.

35 Slide 35 Set Up Notifications Define thresholds Apply thresholds to targets Set up notifications Set up for notifications Create rules and rule sets You ve defined your thresholds for key middleware performance metrics in the Setting Operational Goals stage by creating monitoring templates. And you ve applied these thresholds to your targets using template collections. Now you need to set up notifications so you ll be notified when your warning and critical thresholds are crossed. This allows you to be proactive in your middleware management and monitoring. You ll know about a problem before it affects users negatively. For example, when SOA applications are failing, proactive alerting lets you initiate the diagnostic process before your phone starts ringing. To set up notifications you need to perform some technical setup tasks and then you need to create rules and rule sets. Let s look at these in more detail.

36 Slide 36 Types of Notifications / page Administrator OS command PL/SQL procedure SNMP SNMP third-party applications Notifications are typically performed by sending an and or a page to an Administrator as you see here. For example, you may choose to an Administrator if a warning threshold is crossed whereas you page them if a critical threshold is crossed. But you can also set up more advanced notification methods. You can execute operating system commands (including scripts), you can also execute PL/SQL procedures, and you can send SNMP traps to SNMP-enabled third-party applications. So, for example, if a critical WebLogic Server goes down, you may want to automatically open an in-house trouble-ticket using an operating system script so that the appropriate IT staff can respond in a timely manner.

37 Slide 37 Set Up for Basic Notifications / page Administrator: Set up Mail Server Define Administrators addresses To set up for basic notifications, that is and page, you need to set up your mail server and define addresses for each administrator who needs to receive notifications. And for notifications, you can customize the format. You can add target properties such as Line of Business, Owner, Contact, etc in the to provide additional operational information that can be very helpful when a notification occurs. More set up is required for the advanced notification methods which is beyond the scope of this course and I won t go into here. Then to start receiving notifications, you create rules and rule sets. But before we go into that, let s talk about the different kinds of occurrences that can trigger notifications.

38 Slide 38 What can Trigger Notifications? Performance Space Target Down INCIDENTS Metric Alerts Job Events Standards Violations EVENTS Availability Events Other Events As you now know, when a threshold is crossed this creates a metric alert. A metric alert is one type of event in Cloud Control. An event is a significant occurrence on a managed target that typically indicates something has occurred outside normal operating conditions. For example, in addition to threshold violations creating a metric event, a database target down creates an availability event, a job failure creates a job event, a compliance standards violation creates a standards violation event, and so on. Cloud Control monitors the software stack from applications, databases to hosts and the operating system. When Cloud Control detects issues in any of this infrastructure, it raises events. You can manage events using incidents. Generally, an incident is a situation or issue you need to act on. Of all the events raised within your managed environment, there is likely only a subset that you need to act on because they impact your business applications. An incident is, therefore, a significant event (such as a target down event), or a combination of events that relate to the same issue (for example, running out of space can be detected as separate events raised from the database, host, and storage targets). You manage incidents using Incident Manager which we'll discuss later. You can set up notifications to be triggered by either events or incidents.

39 Slide 39 Create Rules to Automate Notifications Targets Rule Set Notifications Administrator So now you need to automate the notifications related to events and incidents. This automation in support of operational processes is an important part of any scalable monitoring solution. To do so you create rules and rule sets. These enable you to choose the targets and the conditions for which you want to receive notifications from Cloud Control. Basically, a rule is an instruction for Cloud Control to take action on an event, incident, or problem. (A problem is a special type of incident raised by Cloud Control that is outside the scope of this course so I won t cover them here.) As we saw earlier, actions include sending and page notifications, opening helpdesk tickets, creating incidents, and so on. To create rules you create rule sets. These allow you to logically combine different rules that operate on a common object such as a group of targets, as you see here, into a single manageable unit.

40 Slide 40 Rules Example Rule Set for PROD-GROUP Applies to: PROD-GROUP Target Down Rule Criteria: SOA Composite Average Response Time, Business Faults metric alert events of warning or critical severity Action: Create incident, Set Priority = High and Pager Rule Criteria: All Incidents with severity= Critical or Warning Action: If severity = Warning, If severity = Fatal or Critical, page Escalation Rule Criteria: All Incidents open > 24 hours Action: Escalate to Level 1 For example, you can create a rule set for your production targets, that is, all the targets in your Production Administration Group. You can then create the rules. You can set a rule on a number of metric alerts, say, if SOA composite 'Average Response Time' and 'Business Faults' cross warning or critical thresholds then an incident is created. Oracle recommends monitoring and managing events using incidents. Therefore use rules to automatically create incidents for events that are important to be managed. In a rule set, the rules that create incidents should typically be the first set of rules in the rule set. There is an outof-the-box rule set that does this which you should review to see if it meets your requirements. Then you can create another rule on incidents of warning or critical severity. If the severity is warning you can be notified by , if the severity is critical you can be notified by page. Finally you can create another rule on incidents with critical severity. If they are open more than 24 hours you can escalate the incident to level 1. So this rule set automates a business process to create an incident, notify you of the impending problem and escalate it if no action is taken.

41 Slide 41 PROPERTIES Allow user to leave interaction: Show Next Slide Button: Completion Button Label: Anytime Show upon completion Next Slide Here are a few tips on setting up your rules. Click on each of the segments within the circle to view Oracle s recommendations. Rules Recall that you create rules to automate the notifications related to events and incidents. Plan You can see setting up your rules is very important so take time to plan the rule structure before implementing them. Use Groups Specify an Administration Group as the object of the rule set rather than individual targets. It s much easier to manage and monitor your targets in groups. The group you use in your rule set should contain targets that have common requirements for notification as well as common requirements for actions on events and incidents. As the group later expands and additional targets are added, the rule set will automatically apply to the newly added members without further modification of the rule set. Also put all the rules that pertain to an Administration group into one rule set instead of across multiple rule sets. Doing so makes it much easier to track and manage all actions on the group because it s centrally defined in one place. And it avoids potential duplication of actions across rule sets. Create Rules Set the rules for production groups different than the rules for non-production groups. Your production operations are critical and so you should be notified immediately if anything is going wrong before it becomes a major problem. Non-production targets are not as critical so you don t need to have such stringent requirements. Create Incidents Create incidents for the events specified in the rule. Since there are operational requirements to send notifications for these events, then this means these events are actively managed and

42 thus incidents should be created for these. Rules that create incidents for events should typically be the first set of rules in the rule set. Act on Incidents Once incidents are created, perform any other subsequent actions in the rule set, such as notifications, on the incident instead of on the event. One exception would be in the case where interested parties are interested in receiving notifications for specific events. For example, business owners might want to be notified of target down events for targets that impact their business applications. Remind Set up reminders using repeat notifications so administrators are notified repeatedly until an important incident is acknowledged. When critical thresholds are crossed you need to ensure the problem is taken care of by an Administrator. Escalate If notifications are not being responded to, for example, the recipient may be out sick for a day, escalate these unattended, important notifications. Send a page to a different person, say at the manager level, if an incident is open too long. Review If you are receiving too many or not enough notifications, review your thresholds. You may need to fine-tune threshold values over time until your system meets or exceeds your performance goals. You can use as a guide recommendations made by Cloud Control for thresholds based on metric data for recent periods of time, for example the last 31 days.

43 Slide 42 Create Reports You ve set up notifications to be notified when things are changing. However, you may still want confirmation that your metrics are good. Cloud Control comes with a build in reporting infrastructure that you can use to generate reports. So to check that everything is OK, generate reports on your managed targets. For example, you may want to run weekly reports on the performance of your critical WebLogic Server, SOA or OSB performance. The report shown here shows a Summary, Availability State and Availability history for a host target over the past 24 hours. You can use predefined report definitions to generate fully formatted HTML reports, which show critical operations and business information, without any additional configuration or setup. Or you can make your own customized reports that suit your specific operational needs. And you can set up automatic delivery so you don t need to run them manually.

44 Slide 43 Customize and Build Monitoring Dashboards Proactively monitor a few summary dashboards Customize default dashboards Create new dashboards It's also a good idea to have at least a few Cloud Control summary dashboards that are targeted to your critical applications that you can proactively monitor. These can give you a quick overview of the health, status, and performance of your middleware targets and Cloud Control provides many default dashboards to do just that. However you should customize these and create new ones to meet your specific monitoring requirements for your mission critical applications. This lets you more efficiently analyze, correlate and diagnose performance problems.

45 Slide 44 Customize and Build Monitoring Dashboards WebLogic Server Home Page Middleware Performance Summary SOA Home Page Composite Application Group Home Page Incident Views Here you see some of the most useful dashboards to customize or build. Click on each of these dashboards to find out more details. When you are finished exploring, click Next to continue on with the course.

46 Slide 45 Customize WebLogic Server Home Page Home pages make it easy to locate the most important monitoring data and the most commonly used administrative functions. However, you may want to go beyond the default views to see monitoring data particular to your environment. You can customize any target home page in Cloud Control but two important home pages for Middleware management and monitoring are the WebLogic Server Domain and the WebLogic Server home pages. The default view, as you see here, gives you an overview of the state of your WebLogic Servers but you may want to see additional metrics, for example, you can add particular metrics you are interested in using the Performance Metric Chart. You can also reorganize the regions displayed to help you monitor your servers more easily. Do this by personalizing the page - you can customize the format of the page as well as choosing what data to display on the page, taking off some content and adding other content.

47 Slide 46 Customize SOA Home Page The SOA target home page is the main landing page and jump-off point for users to interact with Cloud Control functionality related to the Oracle SOA Suite. Here you see a comprehensive overview across the entire SOA Infrastructure - aggregated throughput metrics for the whole soa-infra, summary metrics for the service engines, and summaries for the SOA composites. You can use this screen to get a quick overview of the health, status, and performance of the SOA domain. Add specific additional metrics and re-organize the regions which are displayed to suit your particular needs.

48 Slide 47 Customize Group Home Page An important home page worth customizing is the Group Home Page. When you create an Administration group to group together all your targets for a particular purpose, for example, all your production targets or all your WebLogic targets associated with a particular set of applications as you see here, you get a group home page so you can visually monitor them together. Customize this home page so you see just the monitoring information you need for your particular requirements.

49 Slide 48 Customize Middleware Performance Summary An important Middleware monitoring dashboard you can customize is the Performance Summary page. Here you see all the relevant performance data for a particular middleware target. Using this page you can ensure your Fusion Middleware and WebLogic Server deployments are performing as expected and meeting your targets. You can customize this page to monitor exactly the performance metrics you want. Here we've added 'Deployment Work Manager Requests' to the default view. You can also select multiple metrics at a time for correlation, overlay metrics for comparison and better analysis, compare performance impact on a periodic basis, and create performance summary baselines which is like a snapshot of performance that can be used as a benchmark.

50 Slide 49 Create Composite Application Dashboards As well as customizing existing dashboards in Cloud Control you can create new ones. In complex datacenter environments administrators need a single view of top level metrics and information related to critical business services and applications. Composite Applications are a way for you to create your own dashboards for your custom applications. This lets you build a comprehensive view representing a multi-tier composite application composed of multiple application deployments and SOA composites. You can easily include all additional components (such as databases, service buses, Coherence clusters, and other middleware and nonmiddleware targets). The Composite Application dashboard provides full visibility across the composite application with access to key monitoring and diagnostics regions, which can be easily customized and personalized. The overall result is a single dashboard view providing not only health information about the application, but also deeper visibility into component health and incidents at a glance. Each logical application environment in your enterprise can have its own composite application dashboard and these dashboards are fully customizable. You can personalize any of the regions to display any relevant metric.

51 Slide 50 Create Incident Views As well as monitoring performance metrics on various dashboards, you want to monitor your incidents by creating incident views. Now that monitoring has been set up, events will be raised and incidents created for these (using the rules you created). Incident Manager, what you see here, gives you a centralized way to manage all these different incidents. It s a central location from which you can view, manage, diagnose and resolve your incidents as well as identify, resolve and eliminate the root cause of disruptions. On the left side are views that allow you to look at specific incidents of interest. Cloud Control comes with default views for the most common tasks, for example, there is a view for all your open incidents and problems, another for unassigned incidents, etc. Create your own custom views to filter incidents based on your specific search criteria. A good example of this is to create a view that shows all incidents for an important group of targets, for example your Production Group.

52 Slide 51 Monitor Diagnostic Findings Metrics Configuration Settings Another monitoring task you should perform regularly is to check for Diagnostic Findings on your WebLogic Server targets. The Middleware Diagnostics Advisor takes advantage of having all the configuration information for servers, applications, and hosts as well as all the performance metrics and component dependency data that Cloud Control collects automatically. It analyzes the entire stack of targets from the WebLogic Server down to the database. Based on this information it generates findings, displaying all of this correlated performance management data on a single console. More importantly, instead of just providing raw metrics and configuration details, it helps you to resolve problems and improve performance by giving actual advice. For example, it can help you identify slow SQL statements as you see here or that a JDBC connection pool is causing a performance bottleneck. We'll talk more about diagnosing problems later but you can also use the the Middleware Diagnostics Advisor page to locate a potential problem by focusing on the most common finding based on the number of occurrences in the past 6 and 24 hours. Examine the summary information at the top of the finding, including the charts. Also examine the data in the Analysis Information table to understand the finding better, and also examine the related targets table to determine which targets are impacted by this finding. Respond proactively by performing the action referenced in the Suggestion column. Here it suggests tuning the SQL statements might improve performance. Note, that currently, there are no SOA specific findings for the Diagnostics Advisor.

53 Slide 52 When to Monitor Diagnostic Findings Monitor findings during: Peak load time High CPU utilization High disk I/O High memory utilization So when is it advisable to monitor diagnostic findings? First, monitor the findings during peak load time to check for potential problems that the load might have caused. Also, monitor findings during high CPU utilization, high Disk I/O, or high memory utilization to check for problems that might have caused the high utilization.

54 Slide 53 Monitor Transactions Using Business Transaction Management Lastly, use Business Transaction Management to monitor your transactions. The operational health summary dashboard gives you a roll-up of key metrics for your system and transactions. The indicators will quickly tell you if something is wrong, and the associated number will tell us the actual number of issues. For example, here you can see that one service and one endpoint are down.

55 Slide 55 Monitoring Proactively Summary Generally, manage by exception Associate Template Collections with Administration Groups Set up notifications Create reports Customize and build monitoring dashboards Monitor Diagnostic Findings Monitor transactions with BTM Now let s summarize the best practices we covered in this monitoring proactively stage of the development lifecycle. Generally, Oracle recommends to manage by exception. Associate template collections with Administration Groups to quickly apply standardized monitoring settings to many targets of different types and have them applied automatically if a new target is added to the group. Set up notifications by creating rules and rule sets so you ll be notified of impending problems. Create reports to confirm that your metrics are good. Customize and build monitoring dashboards to meet the specific monitoring requirements of your mission critical applications. Monitor diagnostic findings during peak load, high CPU utilization, high disk I/O and high memory utilization times. Monitor transactions with BTM to view the operational health of transactions and the systems involved in those transactions.

56 Slide 56 Road Map Plan Improve Processes Set Operational Goals Track & Control Changes Monitor Proactively Diagnose Problems You need to diagnose problems at all stages in the development lifecycle through Development, Test, QA and production, but it s vitally important in production. When your critical business applications go down or are not performing well, your business suffers. Here I ll give you best practices on diagnosing problems quickly and efficiently.

57 Slide 57 Diagnosing Problems Best Practices Narrow down the problem Drill down to find the root cause View Diagnostic Findings Create Diagnostic Snapshots Diagnosing problems is a very challenging area as there can be so many different types of problems with so many different causes. And middleware applications introduce a new order of complexity in tracking down failures as they are very complicated with multiple tiers within the application stack. There's no one master methodology to finding the root cause of a problem as it depends on the symptoms you re experiencing. So for this course, I ll be covering diagnostics at a very high level and making general recommendations about how to use Cloud Control to solve your middleware performance problems. Oracle s recommended approach is to first asses your notifications and/or user reports to narrow down the possible cause of the problem to a particular part of the infrastructure, for example, is it a problem with a WebLogic server, the SOA infrastructure, or a particular SOA composite? Then, once you ve narrowed it down, drill-down using the appropriate customized dashboard to find the root cause. You should also view diagnostic findings from the Middleware Diagnostics Advisor. And, if you re in the middle of a critical outage but don t have time to diagnose the root cause, create diagnostic snapshots for later root cause analysis.

58 Slide 58 Narrow Down the Problem 1. Assess notifications / user reports 2. View customized summary pages So first you need to narrow down the possible cause of the problem to a particular part of the middleware infrastructure. Typically you find out about a problem by either receiving an alert notification or getting a phone call from someone. This initiates the diagnostic process. Ideally alert notifications are received before the problem has escalated too far and users are starting to notice. When you get one or more notifications, you know what metric thresholds were violated so this should help you to narrow down the cause and guide you to where to start looking. For example, notification of excessive CPU consumption by a WebLogic Server would lead you to view the WebLogic Server's home page to see what applications are running on that instance and investigate further. When you are notified of a problem by a person, this can be more challenging to narrow down as the symptoms they are seeing, for example, slow response time of an application, could be caused by so many factors, many outside of the middleware layer, for example network issues. For this course we'll assume all non-middleware infrastructure issues have been ruled out. So if you don't have a clear idea of where to start, this is where your high-level customized dashboards come in useful. For example if the complaints center on the performance of one SOA composite application you can view your composite application dashboard that you created which will show all the composite application dependencies in real time. You then may be able to see that the problem is in an underlying WebLogic Server so you can then investigate further. So the notifications and/or user reports help you to form an initial hypothesis. Then you want to view the appropriate customized summary page in Cloud Control. These provide a view from the 4,000 foot perspective and they should point you in the direction of the problem.

59 Slide 59 Drill Down to Find the Root Cause 1. Assess notifications / user reports 2. View customized summary pages 3. Drill down into problem area Then from your customized dashboards, drill down into the problem area to find the root cause of the problem. This root cause analysis is handled by drilling down through multiple views that are automatically discovered by Cloud Control. These detailed models of the application infrastructure allow you to maintain your context throughout the entire drill-down process from multiple entry points ensuring that you are able to quickly get at the application metrics required to quickly resolve a performance bottleneck or other application issue. The goal is to resolve the problem as fast as possible with as little impact on the overall application as possible.

60 Slide 60 Drill Down from Summary Dashboards Target Home Page Incident Management Console Middleware Performance Summary Composite Application Application Dependency & Performance Business Transaction Management Log Viewer You can drill down into deeper diagnostics from just about any page in Cloud Control. Here you see some of the most useful dashboards you can use to drill down into specific components to find the root cause of a middleware problem. Some of these you ve seen before in this course. Click on a dashboard to find out how you can use it to drill down for root cause analysis. When you are finished exploring these dashboards, click Next to continue on with the course.

61 Slide 61 Drill Down From Composite Application Dashboard You ll recall that the Composite Application dashboard, which you can create, provides full visibility across all components from the applications to the middle-tier, hosts, services, and databases that those applications rely on. This is a useful dashboard to drill down from for problem analysis. Let s look at an example. Here you can see a composite application comprised of multiple SOA and Java EE components that we are monitoring from a central dashboard. You can see a variety of metrics selected from each of the members including the WebLogic Server and database targets. In addition, you can see the health of the overall members and a specialized thread state table on the JVMs. You can see that some of the JVMs have a lot of locks. The database waits are somewhat concerning and it makes sense to explore at least the health of the JVM. A large number of locks can be a huge red flag that your application is having problems. As the JVM is the most likely culprit, we ll drill in from that perspective while still keeping the context of the overall Fusion Middleware and WebLogic Server target states. If we click on this MedRecServer JVM we can drill down in-context to the JVM to examine its health in more detail. This takes us to the JVM Performance Diagnostics page which shows both JVM and related database data. We ve scrolled down to view the general tab where we can see Top Requests. In this case, we re concerned with our application health, so let s filter on a key request such as one of the patient registration requests which is a requirement of our application. This will filter the entire page by the request allowing us to examine everything in context including all of the thread states. Now on the Threads tab, we can analyze our thread state over the period in question and also look at key categories in relation to those threads such as the thread state of method and request calls. At this point, we see a lot of stuck threads and in particular a lot in the Database Wait state which could indicate a database problem. Let s click on the Live Thread Analysis link to determine how threads are impacting each other live.

62 Let s explore this by clicking on one of the threads to drill into the root cause of why it may be stuck in a Database Wait state which is a definite indicator of a hung application. We can see the thread stack, browse local variables, and examine how it impacts other threads. However, in this case, we want to determine the root cause of why it s stuck in the Database Wait state in the first place. Therefore, let s drill down to the actual database session that is stuck by clicking on the DB Wait link. This takes us to the database session where we can see all the details of the session that s being blocked or running extremely slow. Assuming Cloud Control is monitoring your WebLogic Server, JVM, and database, you can do this type of cross-tier analysis in production environments with less than 1% overhead so it won t impact the applications themselves. In this case, we can now see that the session is stuck within a table lock contention state. Let s click on the Blocking ID to determine which database session is blocking our applications session. We can see that our composite application s database session was blocked by a database session kicked off from SQL Plus outside our application. This likely represents a maintenance routine or another process or application outside of ours. We can drill in even further, to see exactly what the session is doing or we could kill the session that is blocking ours (assuming that we had the necessary DBA privileges) or notify the appropriate DBA to take action and resolve the problem. This cross-tier analysis between the JVM diagnostics and the database diagnostics allows issues in relation to JDBC calls to underlying databases to be easily diagnosed without all of the finger pointing that sometimes occurs when trying to troubleshoot issues between the middleware and database tiers.

63 Slide 62 Drill Down From Middleware Target Home Pages The middleware target home pages are used to get a quick overview of the health, status and performance of your WebLogic and SOA targets. You can also use these pages to drill down to find the root cause of problems. For example, from the WebLogic Server home page you can easily drill into key metrics associated with the JVM of your applications in the context of the WebLogic Server domain you are analyzing for performance trends or a potential bottleneck. Here s another example. You receive an alert notification of excessive CPU consumption by WebLogic Server. This leads you to investigate the applications running on that instance. By using the Servlets and JSPs tab in the Most Requested section of the WebLogic Server Home page as you see here, you can quickly identify the highest volume or least responsive application. Then you can drill down and diagnose the application's servlets, Java Server Pages, or EJBs to identify the bottleneck. And, if necessary, you can drill all the way down to the JDBC/database layer. Even if you re unfamiliar with the application design, you can quickly pinpoint problematic bottlenecks and analyze the metrics and architecture associated with them in order to facilitate a quick turnaround to get the application up and running at the required service level.

64 Slide 63 Drill Down From Middleware Performance Summary Performance Summary pages are where you see all the relevant performance data for a particular middleware target. We looked at customizing these pages earlier to show just the metrics you re interested in. Here you can drill down to help diagnose and resolve problems. If you click on the metric name, you get an Additional Information popup. From here you can view a Problem Analysis page which shows other related infrastructure metrics which are affecting the metrics being analyzed. Inspect the charts for unusual increases in recorded metrics such as request processing time, CPU usage in storage units or percent utilized, number of requests per minute, Java Virtual Machine heap memory usage, or server memory usage. If you find that request processing time increased due to a high number of requests per minute, you may need to increase the capacity of your system. If the metric charts do not indicate the cause of the problem, scroll down to the Related Information pane and inspect the List of Related Targets To Analyze table for information about target health (status) and recent configuration changes. If you want to see the topology of the components for which data is being displayed, click the Related Targets Topology tab. If the List of Related Targets to Analyze table does not indicate the cause of the problem, scroll up and click a link adjacent to one of the metrics and then click Analyze Logs in the Additional Information pop-up to display log messages for the selected target and its members during the selected time period. When used together, Problem Analysis and Analyze Logs can help you inspect metrics, target status information, and logs during troubleshooting.

65 Slide 64 Drill Down From Incident Management Console Typically you will have incidents created automatically when your critical thresholds are crossed. From the Incident Management Console you can view and manage your incidents. You can also diagnose and resolve incidents here. It s even integrated with My Oracle Support for accelerated resolution of problems. Earlier you created views to filter your important incidents. Now you can zero in on the critical ones and drill down to resolve the problem. In this example, I created a Production Incidents view, and here we need to investigate the SOA composite that is down. When we select this incident, detailed information is show in the bottom portion of the screen. In the bottom-right portion of the page we see links to the guided diagnostics and resolution area to assist you in resolving the issue. For example, if I click the View Topology link I see the topology view where I can see the health of other targets the SOA composite depends on. And I can further drill down, in-context, into relevant lower level metrics to further investigate performance behaviors.

66 Slide 65 Drill Down From Log Viewer The Log Viewer lets you see critical, or even just suspicious, errors across all log files, including WebLogic and Fusion Middleware logs. So you can gain access to log files regardless of where they reside or what product they relate to. Here you can search and correlate messages across log files based on time, severity or Execution Context ID (ECID). Then you can use the ECID to drill down to the middleware tier for deep-dive details to help you diagnose and resolve problems.

67 Slide 66 Drill Down From Application Dependency & Performance UI The Application Dependency and Performance, or ADP, pages within Cloud Control analyze SOA, Oracle Service Bus, Portal, and ADF applications to capture the complex relationships among the various application building blocks. ADP delivers a holistic, service-oriented view across heterogeneous environments. Here we see the ADP page for a specific Oracle Service Bus proxy service. We can see a summary of the proxy service s performance in tabular and in graphical form, as well as a summary of the performance of each of the pipeline nodes. We can further drill down and see individual pipeline nodes within the proxy service allowing us to diagnose problems within the service.

68 Slide 67 Drill Down From Business Transaction Management Business Transaction Management, or BTM, gives you a central view of distributed transactions that you can drill down into. For example, two useful predefined dashboards are the Top 10 Services and Top 10 Transactions in terms of load, uptime issues, response time and faults. Here you see the Top 10 Services dashboard. These dashboards are good starting points for further analysis, since they give you a good view over the worst performers. Then you can drill into transaction content and context. This gives a unified view of transactions even across different keys, IDs, etc. Let's look at resolving an example problem with BTM. A customer, Epsilon Enterprises, has called in, complaining about some orders that don't seem to be going through and are getting lost somewhere. Since we don't know exactly where the issue is, but we do know the customer name, we'll start by searching through our messages using the customer name. We can see a few messages related to this customer and a couple stick out as having a much longer response time than the others. Let's take a look at one of them. Now that we've found a suspicious message, we can use this to look at the relevant transaction. In the transaction inspector, we can see that something is wrong with the PurchasingService. Drilling down into this service, we can now see in the response that something is wrong with the outbound service endpoint URL, most likely a typo in the service name.

69 Slide 68 View Diagnostic Findings from the Middleware Diagnostics Advisor Earlier we talked about monitoring diagnostic findings from the Middleware Diagnostics Advisor to locate potential problems. You should also consult the diagnostic advisor real-time to resolve current problems quickly and with guided help. For example, on this WebLogic Server home page, we see Diagnostic Findings which is a clear indicator that Cloud Control has discovered a key issue with the overall server by analyzing the performance and configuration data of the entire stack including the middleware and database. If we click the Diagnostic Findings link, this starts the Middleware Diagnostics Advisor. We can see that there are at least two findings. One related to locked threads and one related to the SQL execution under the covers of our Medrec application. We can see these occurred a number of times with an hourly interval indicating it s a regular problem that s occurring constantly as opposed to being intermittent. Let s take a look at the details of the findings for this hour. If we click on the SQL finding, we see the details associated with the finding and we can see the associated metrics and configuration data that helped determine the cause of this finding. You can see that it s found that SQL execution is taking a long time causing requests to pile up and increasing response time and reducing throughput. And you can also see the recommendation which is to tune SQL statements to improve performance. It also lists the SQL statements themselves which are taking a significant time to run, in this case just one. You could even drill down into the top one which is the slowest, to look at the SQL in the SQL Analyzer which would allow the DBA to analyze it more fully and tune the SQL. If we look at the Thread Lock finding, we can see that there are threads in the WebLogic Server that are holding locks for a long time and other threads that are waiting and can t proceed until these locks are acquired. This is causing server performance to degrade. The recommendation is to drill down into JVMD diagnostics by clicking on the Lock object to determine the threads holding and waiting on these locks. And it also recommends to improve the design of the code to prevent these situations from occurring.

70 So the Middleware Diagnostics Advisor with it s proactive root-cause-analysis helps you save valuable time when it comes to identifying performance bottlenecks in production applications.

71 Slide 69 Diagnose Problems Using Diagnostic Snapshots Quickly capture information about a problem. Share snapshots with Oracle Support and colleagues. Problems often arise suddenly and need to be remedied immediately with no time to capture the details for later analysis of the root cause. For example, your WebLogic Server and the underlying JVM enter some locked condition which causes your end users to hang. You need to get your users up and running as soon as possible, and so you don t have time to figure out why the problem occurred. You would like to keep data on the problem, and diagnose the issue later, in case it happens again. In these situations, you should immediately capture the state of the JVM, overall metrics for the WebLogic Server, and log data, as the problem occurs, into a single packaged diagnostic snapshot for later analysis. These diagnostic snapshots can then be exported and shared with others to help diagnose the root cause of the problem. Oracle Support can also use these snapshots to diagnose problems, allowing you to communicate your problems to support much easier. You can also use these snapshots to validate that a fix you put into place for a problem has in fact resolved it. This allows you to build in preventatives to better protect your environment against outages and bottlenecks in the future similar to the one that impacted your system. Note that currently there are no SOA metrics in the Diagnostics Snapshot.

72 Slide 71 Diagnosing Problems Summary Assess notifications/user reports to narrow down the problem Drill down using customized dashboards View Diagnostic Findings Create Diagnostic Snapshots Now let s summarize the best practices we covered in this diagnosing problems stage of the development lifecycle. Assess notifications and/or user reports to narrow down the problem to a particular part of your enterprise infrastructure. Drill down from the appropriate customized summary dashboard to find the root cause of the problem. View Diagnostic Findings from the Middleware Diagnostic Advisor to resolve problems quickly with guided help. Create Diagnostic Snapshots in the midst of resolving problems for later root cause analysis.

73 Slide 72 Road Map Plan Improve Processes Set Operational Goals Track & Control Changes Monitor Proactively Diagnose Problems Tracking configurations is probably one of the most time-consuming and difficult tasks you, as an administrator, face on a daily basis. For example, do you know how each and every one of your managed targets is configured and whether they comply with your corporate or regulatory standards? Here I'll give you best practices on tracking and controlling configuration changes.

74 Slide 73 Tracking & Controlling Changes Best Practices View configuration history Compare configurations Create Gold Standard Configuration Snapshots Customize Comparison Templates Schedule comparisons with notifications Cloud Control automatically collects configuration information for all the hosts and the managed targets on those hosts that have a running management agent. Using this data it lets you easily track and control configuration changes in your middleware environments. As part of configuration management you should view configuration history to pinpoint why a system may be behaving differently today that was working well yesterday. Or compare configurations of two targets that were identical to check for configuration drift. You should also create Gold Standard configuration snapshots that you can compare to your live configurations. And you should customize configuration comparison templates to specify what to ignore and what to alert on. And finally you should schedule your comparisons so they run regularly and set up notifications so you're notified automatically of important differences. All of these extensive configuration management capabilities that Cloud Control provides facilitates compliance with various regulatory and business standards. So let's look at each of these in more detail.

75 Slide 74 View Configuration History Having recent configuration changes for a particular Middleware target readily available is very valuable. For example, when an application that was performing well yesterday isn t performing well today, often, the performance degradation can be attributed to a recent configuration change. So if you notice a sudden change in performance, check the target home page for any Configuration Changes, as you see here. Clicking the link will show you all the configuration changes in the past week, including the old value and the new value. In this example here, we see that the login timeout setting changed from 25 seconds to 27 seconds in the past week. This can be invaluable information to help resolve problems quickly.

76 Slide 75 Compare Configurations STAGING PRODUCTION ThreadCount MaxCapacity StatementCacheSize Another way to figure out why a particular middleware target is not performing as well as it was, is to compare its configuration with another target of the same type. Cloud Control s Comparison wizard allows you to compare configurations to quickly and easily pinpoint the differences between them. This helps to keep systems synchronized and to reduce configuration drift. In addition, it simplifies investigations into why systems that are presumed to be identical, are behaving differently. A common use case for this is when you are comparing a WebLogic Server in a staging environment with another WebLogic Server in a production environment. In this scenario, there s a performance problem with the production environment but not the staging, and you want to compare the configurations as, originally, the two configurations were identical. Upon comparison you discover that a configuration change was made that introduced the poor performance in the production environment. For example, a key tuning parameter such as ExecuteQueue ThreadCount, JDBC Connection Pool MaxCapacity or StatementCacheSize may have been changed. And when configuration differences are detected between targets that were presumed to be identical, you can use Cloud Control to quickly provision configurations such that there are no longer any differences. We ll look at doing this in the next section of the course. Note, that as well as comparing WebLogic Server, domains, and cluster target types, you can also compare other middleware targets such as SOA composites, SOA Infra, JVMs, and more.

77 Slide 76 Create Gold Standard Configuration Snapshots Create snapshots of a well configured target Compare current configurations to Gold Standard configurations to: Monitor drift Quickly identify changes during critical outages An important best practice is to create gold standard point-in-time configuration snapshots for later analysis and comparison. For example, save the configuration of a WebLogic Server you know is configured correctly for your production environment with a label such as Gold Standard Production WebLogic Server Configuration. Then at any time you can compare the current configurations of any of your production WebLogic servers with this saved configuration to determine if any configuration drift has occurred from the gold standard. This ensures that your WebLogic Servers are all configured consistently with this known reference. In fact, your IT management may request that such configuration consistency be checked on a regular basis and a summary generated that identifies any important differences. Also during critical outages, you can compare current configuration settings of the target to the gold standard and quickly identify changes that could be the root cause.

78 Slide 77 Customize Comparison Templates When you perform configuration comparisons, use Comparison Templates. These allow you to specify which property differences to ignore, and which property differences should trigger an alert. Out-of-the-box, Cloud Control provides comparison templates for WebLogic Server, domain, and cluster target types as well as SOA Infra, JVM, and more. These out-of-the-box templates can be used as is, or as a guideline that you can customize. For example, here I m customizing the default SOA template. It s already set up to ignore differences between URLs in the different environments as these are expected to be different across SOA targets. But I may be interested in tracking any changes in Audit Level from the standard so I ll check that to be notified on any differences. So, you should review the out-of-the-box comparison templates and customize them as necessary to meet your specific requirements. In most cases, you may only need to make a few tweaks to get what you need.

79 Slide 78 Schedule Comparisons with Notifications Schedule comparison + = Enter address Automatic notification Also when you perform configuration comparisons you can schedule the comparison to be run immediately or performed later. It s a good practice to set up these comparisons with your Gold Standard configuration to be performed on a regular basis, for example, at the end of each week or the first of the month. This may even be a requirement of your IT management. In addition, when performing configuration comparisons you can enter notification addresses for who to notify if differences are found. You should enter your address (as well as any other administrators) so you can be automatically notified. So by scheduling the comparison job regularly and entering addresses you can be automatically be notified on a regular basis if any configuration drift occurs in your live configuration from your gold standard. Going further, you can also validate your target configurations against industry best practices, Oracle recommendations, and internal best practices by using the Cloud Control compliance standard framework. However, I won t be going into that here as this is beyond the scope of this course.

80 Slide 80 Tracking & Controlling Changes Summary View configuration history Compare configurations Create "Gold Standard" configuration snapshots Customize comparison templates Schedule comparisons with notifications Now let s summarize the best practices we covered in this tracking and controlling changes stage of the development lifecycle. View configuration history to pinpoint why a system may be behaving differently today that was working well yesterday. Compare configurations to check for configuration drift. Create Gold Standard configuration snapshots that you can compare against your live configurations. Customize comparison templates to specify what to ignore and what to alert on. Schedule regular comparisons with automatic notifications of important differences.

81 Slide 81 Road Map Plan Improve Processes Set Operational Goals Track & Control Changes Monitor Proactively Diagnose Problems You probably spend a large amount of time performing various lifecycle management operations - such as installing, patching, and configuring WebLogic servers, as well as deploying Java applications and SOA composites, and so on. These are very important tasks but can be very time-consuming and error-prone. Here I ll give you best practices on improving your processes and automating common lifecycle management operations. This will allow you to perform these tasks much faster and more reliably.

82 Slide 82 Improving Processes Best Practices Clone Middleware components from the Software Library Create Gold Images Create WebLogic Provisioning Profiles Customize default Deployment Procedures Automate the application of WebLogic Server patches Rather than installing or deploying Middleware components manually, which can be error-prone and introduce inconsistency in your environment, you should clone them using Cloud Control. And the best way to do that is to clone from Cloud Control s Software Library. This ensures consistent, standardized images and components are deployed across your environment. You should also create gold images of tested Middleware Homes and SOA Artifacts in the software library for later provisioning. As well as creating WebLogic Domain provisioning profiles to act as templates for WebLogic domain provisioning. You should also customize the default Cloud Control deployment procedures to meet your unique provisioning requirements. And lastly, you should automate the application of WebLogic Server patches to make this whole process simple, reliable, fast, and one you can do without any downtime! You can automate all these routine tasks using Cloud Control s Software Library and Provisioning Framework. This lets you improve quality of service by minimizing downtime due to configuration change. And it reduces your IT operational cost through automated deployment procedures to provision software. Let s look at each of these in more detail.

83 Slide 83 Clone Middleware Components WebLogic Domain Admin Server WebLogic Domain Manual installations are slow and error-prone Admin Server Cloning is fast and eliminates errors Cluster Cluster Administrators often rely on manual installation methods in order to provision new middleware environments or add capacity to existing servers which support their production applications. However manual installation methods can take too long and are error-prone. A much more efficient means to provision new middleware environments and add capacity is to clone middleware components in Cloud Control. Cloning reduces time and eliminates errors in building environments. You can clone directly from a source domain to create a new domain or scale up or out an existing domain. Scaling up refers to adding managed servers to a domain on existing hardware, while scaling out means adding managed servers to a domain on new hardware. You would add capacity to existing domains or clusters to accommodate an increase in load. For example, if your deployed application is incurring high load and not performing adequately then you need to scale up or out to improve application performance. You can also clone directly from a source domain to deploy Java EE applications, or BPEL processes or from a SOA domain to provision SOA artifacts such as SOA composites, web services policies, JPS policies, and so on. You can also clone the binaries in the Middleware home directory.

84 Slide 84 Clone Middleware Components from the Software Library WebLogic Domain WebLogic Domain Ensure consistent, standardized images are deployed Rapidly Admin scale Server out or revert to known good state Simplify lifecycle management Admin Server Cluster Software Library Cluster You can clone domains and their components directly from an existing domain, but Oracle recommends cloning from Cloud Control s software library to ensure consistent, standardized images and components are deployed across your environment. This also allows you to rapidly provision middleware environments or quickly revert to a known good state. For example, you may need to scale out an application on another environment very quickly after a new version is introduced that fixes a bug or revert an application to a known good state when a problem is encountered. Cloning Middleware components from the software library simplifies lifecycle management. The software library serves as a central repository for metadata and binary content of application software, software patches, reference gold images, etc. Here you can store various revisions of WebLogic Server domains (both the binaries and the domain configuration), Middleware home binaries, Java EE applications, SOA Artifacts, BPEL processes, Oracle Service Bus Resources, Java Platform Security configuration, and so on. You can create software library components in the context of your reference domain. And, when you clone a component, for example a WebLogic domain, you can make changes to the destination domain configuration as necessary, such as host names, ports, and so on. So you should clone components from staging or reference servers into the software library then deploy those components centrally from the software library to new hardware.

85 Slide 85 Create Gold Images Gold images are: Based on corporate standards Compliant and tested Stored in the Software Library Create Gold Images for: Middleware Home SOA Artifacts Gold images are created by component designers based on corporate standards from reference deployments. They are compliant and tested images, stored in the software library, from which they can be used by all operators in a consistent manner. You can create gold images for Middleware Homes and SOA Artifacts. Middleware Home Gold Images contain only the WebLogic binaries. SOA Artifacts Gold Images contain SOA Artifacts such as SOA Composites, Oracle WebLogic Server Policies, Assertion Templates, JPS Policy and Credential Stores. So create these tested gold images in the software library for later provisioning.

86 Slide 86 Create WebLogic Domain Provisioning Profiles Create a Template for WebLogic Domains + = Middleware Home Binaries Domain Configuration Provisioning Profile You should also create WebLogic Domain Provisioning Profiles from existing reference domains. A Provisioning Profile consists of both the Middleware Home binaries and the domain configuration. You create a profile, save it in the Software Library, and then use the saved profile as the source for creating new WebLogic domains. This will ensure that future WebLogic installations follow a standard, consistent configuration. It also saves time and reduces the risks of making mistakes. You can think of the provisioning profile as a template for WebLogic domains provisioning.

87 Slide 87 Customize Default Deployment Procedures Deployment Procedure Deployment Procedure Deployment Procedure Add environment variables Add custom steps Disable unwanted steps Add custom scripts notifications Use authentication tools Use different error handlers To actually perform provisioning you use a deployment procedure. This contains a hierarchical sequence of provisioning or patching steps that need to be performed for a particular lifecycle management activity. Cloud Control comes with a set of default Deployment Procedures that help you accomplish common provisioning and patching-related tasks. These default procedures have been created considering all the best practices in the industry. And each Deployment Procedure is unique, and is designed to perform a particular operation according to the source being provisioned. For example, the Deployment Procedure to deploy a Java EE application differs from the one to deploy a SOA composite. Understandably, your provisioning and patching-related requirements will vary from one environment to another, or from a test installation to a production installation. So use the default Deployment Procedures and customize them according to your needs. For example, you can add environment variables, include additional custom steps, disable unwanted steps, add custom scripts, and use authentication tools to run some steps as another user. You can even implement different error handling methods. And you can reference any of the Middleware entities in the software library to automate the patching, provisioning or deployment of the associated software. To further automate provisioning, set up notifications to report the status of deployment procedure execution so you know immediately if it was successful or not, or action is required.

88 Slide 88 PROPERTIES Allow user to leave interaction: Show Next Slide Button: Completion Button Label: Anytime Show upon completion Next Slide Here you see a list of the default Middleware deployment procedures that you can customize to meet your needs. Click the tabs to your left to get an overview of each deployment procedure. Provision Middleware This procedure clones and configures a Middleware Home and/or WebLogic Domain from the Software Library. Scale Up/Scale Out Middleware This procedure clones existing WebLogic Managed Servers within a domain or adds new managed servers to an existing domain in order to add capacity to the domain or cluster. Use this procedure to scale up on existing hardware within the domain or scale out on new hardware. Deploy/Undeploy Java EE Applications This procedure deploys, redeploys, and undeploys Java EE applications to and from WebLogic domains. You can deploy to multiple servers or multiple domains and you can also specify the archive files to deploy, any pre or post deploy WLST scripts, and so on. SOA Artifacts Provisioning This procedure clones and configures SOA Artifacts such as SOA composites, Web Service policies, Java Platform Security configuration, Human Workflow configuration and B2B configuration from a reference SOA Domain environment or gold image to a destination SOA Domain. Deploy SOA Composites This procedure deploys SOA Composites on a selected SOA domain. The composite and the plan can be selected from a file system accessible from the destination domain host or from the software library. BPEL Process Provisioning

89 This procedure deploys BPEL Processes on a selected Oracle BPEL Process Manager. To do so, select BPEL Process suitcase files which are BPEL process JAR files stored as generic components in the Software Library. Oracle Service Bus Resource Provisioning This procedure deploys Oracle Service Bus resources to an Oracle Service Bus Domain. You can deploy the resources as part of a project or individually. Resources include Business Services, Proxy Services, WS-Policies, WSDL schemas, XQuery transformations, and JARs.

90 Slide 89 PROPERTIES Allow user to leave interaction: Show Next Slide Button: Completion Button Label: Anytime Show upon completion Next Slide Here you can explore the recommended workflow for provisioning WebLogic domains. Click the steps along the bottom to view more details. In this first step, you manually install the WebLogic software, patch WebLogic, configure the WebLogic domain and then fully test the environment until you consider it a reference installation that all future installations should conform to. In this step you create a WebLogic Domain provisioning profile of the reference installation; where the WebLogic binaries and the domain configuration are stored in the software library for use as the source for future cloning operations. In this step you use the provisioning profile to create a customized deployment procedure. Here you can set new inputs, like ORACLE_HOME, ORACLE_SID, and other configuration parameters. This prevents you from having to repeat identical configuration inputs for each deployment procedure. You can also lock the procedure so that any operators running the procedure cannot modify it. This deployment procedure is now a best practice procedure. In this step you use the best practice deployment procedure to clone WebLogic domains with a standard, consistent configuration quickly and without errors.

91 Slide 90 Automate Application of WebLogic Server Patches Patch Plans List patches to apply Specify targets CC performs validation checking Specify deployment procedure Patch Templates Are created from Patch Plan List patches to apply + deployment options Do not specify targets Patching is an important phase of the WebLogic Server lifecycle that enables you to keep your WebLogic Servers updated with Oracle bug fixes. However, patching has always been a very challenging phase of the lifecycle because it is complex, risky, time consuming, and involves downtime. With Cloud Control you can easily automate the application of patches across WebLogic Servers. You can automate applying one-off patches and critical patch updates across WebLogic domains by creating Patch Plans. These plans help you create a consolidated list of patches you want to apply as a group to one or more WebLogic targets. As you create the plan and specify the actual targets, Cloud Control performs prerequisite checks and analyzes the patches for conflicts and potential problems. This automated validation checking minimizes errors. Also when creating the patch plan you specify the deployment procedure to use. WebLogic Server has two default deployment procedures for patching, which of course you can customize to meet our needs. The default plans let you apply patches in either parallel or rolling mode. Applying patches in rolling mode eliminates downtime as the nodes of the cluster are patched individually, that is, one by one. For example, if you are patching a cluster that has five nodes, then the first node is shut down, patched, and restarted, and then the process is rolled over to the next node until all the nodes are patched successfully. In contrast, when patching in parallel mode, all the nodes are shut down and the patch is applied on all of them at the same time. You can save a patch plan as a patch template. This template contains the predetermined set of patches and deployment options saved from the source patch plan, but without the targets specified. This can then be used to create new plans to roll out already tested patches to your enterprise. So automating the application of patches with patch plans and patch templates makes the whole process simple, reliable, and fast, and avoids downtime!

92 Slide 91 PROPERTIES Allow user to leave interaction: Show Next Slide Button: Completion Button Label: Anytime Show upon completion Next Slide Here you can explore the recommended workflow for automating the application of WebLogic Server patches. This process allows you to create predesigned plans based on an existing successfully analyzed and deployable patch plan which you use to select a completely new set of targets. Click the steps along the bottom to view more details. In this first step, identify and download WebLogic Server patches through My Oracle Support which is integrated into Cloud Control. In this step, create a patch plan that contains all the patches you wish to apply and the targets to apply them to. If you re rolling out patches in a mass scale over a period of time, apply the patches on a few WebLogic Servers to test if the patches are being applied successfully. Cloud Control validates the patches can be applied to those targets. Also select the deployment plan to use and then execute the deployment plan to apply the patches. In this step, save the tested patch plan as a patch template which contains everything in the patch plan except the targets. In this step, create patch plans from the patch template to roll out the same set of tested patches to the rest of the WebLogic servers in your enterprise.

93 Slide 97 Improving Processes Summary Clone components from the Software Library Create Gold Images Create WebLogic Provisioning Profiles Customize default Deployment Procedures Automate application of WebLogic Server patches Now let s summarize the best practices we covered in this improving processes stage of the development lifecycle. Clone Middleware components from the software library to ensure consistent, standardized images and components are deployed across your environment. Create Gold Images of tested Middleware Homes and SOA Artifacts in the software library for later provisioning. Create WebLogic Provisioning Profiles to act as templates for WebLogic domain provisioning. Customize default Deployment Procedures to meet your unique provisioning requirements and add automatic status notification. Automate the application of WebLogic Server patches by creating patch plans and patch templates to make the process simple, reliable, and fast and avoid downtime.

94 Slide 98 PROPERTIES Allow user to leave interaction: Show Next Slide Button: Completion Button Label: Anytime Show upon completion Next Slide This self-study covered the most important Middleware management best practices at each stage of the middleware development lifecycle. Click a label for the Best Practices at each stage of the lifecycle. You can also download a summary of the Best Practices by clicking the Attachment link. Plan Deploy Cloud Control universally and then you expand with your business-driven needs. Deploy Application Dependency & Performance where you need deep visibility into SOA, Oracle Service Bus, Portal, and ADF applications. Deploy JVM Diagnostics where you need deep diagnostic visibility into Java applications. Deploy Business Transaction Management where you need visibility into your transactions. And install Real User Experience Insight to monitor and manage your user experience. Set Operational Goals: Define metric thresholds on your critical components based on performance that meets your Service Level Agreements. Create monitoring templates to group together thresholds for targets of the same type. Create template collections to group together thresholds for groups of targets of different types. Create Administration Groups to monitor and manage different types of targets in the same way. Monitor Proactively: Generally, Oracle recommends to manage by exception. Associate template collections with Administration Groups to quickly apply standardized monitoring settings to many targets of different types and have them applied automatically if a new target is added to the group.

95 Set up notifications by creating rules and rule sets so you ll be notified of impending problems. Create reports to confirm that your metrics are good. Customize and build monitoring dashboards to meet the specific monitoring requirements of your mission critical applications. Monitor diagnostic findings during peak load, high CPU utilization, high disk I/O and high memory utilization times. Monitor transactions with BTM to view the operational health of transactions and the systems involved in those transactions. Diagnose Problems: Assess notifications and/or user reports to narrow down the problem to a particular part of your enterprise infrastructure. Drill down from the appropriate customized summary dashboard to find the root cause of the problem. View Diagnostic Findings from the Middleware Diagnostic Advisor to resolve problems quickly with guided help. Create Diagnostic Snapshots in the midst of resolving problems for later root cause analysis. Track and Control Changes: View configuration history to pinpoint why a system may be behaving differently today that was working well yesterday. Compare configurations to check for configuration drift. Create Gold Standard configuration snapshots that you can compare against your live configurations. Customize comparison templates to specify what to ignore and what to alert on. Schedule regular comparisons with automatic notifications of important differences. Improve Processes: Clone Middleware components from the software library to ensure consistent, standardized images and components are deployed across your environment. Create Gold Images of tested Middleware Homes and SOA Artifacts in the software library for later provisioning. Create WebLogic Provisioning Profiles to act as templates for WebLogic domain provisioning. Customize default Deployment Procedures to meet your unique provisioning requirements and add automatic status notification. Automate the application of WebLogic Server patches by creating patch plans and patch templates to make the process simple, reliable, and fast and avoid downtime.

96 Slide 99 PROPERTIES Allow user to leave interaction: Show Next Slide Button: Completion Button Label: Anytime Show upon completion Next Slide More Information Oracle offers a variety of additional channels from which you can learn more about Enterprise Manager Cloud Control or about any Oracle products. We at Oracle Education know your time is valuable and limited, and so we thank you for participating in this self-paced training. We hope this course has met your expectations and learning objectives, and wish you the best of luck in all of your endeavors. Please read through all the tabs on this page to learn about all the Oracle resources available for you. Self-Study Courses Oracle Enterprise Manager Cloud Control 12c Self-Study training is aimed specifically at administrators who are familiar with the current releases of Enterprise Manager. It is designed to bring existing users up to speed on the new features of Oracle Enterprise Manager Cloud Control 12c. The series is broken into a number of tracks, each based around one of the major themes of the new release, and is available from Oracle University. Instructor-led Courses Several instructor-led courses are also included in the Oracle Enterprise Manager Cloud Control 12c curriculum, available through Oracle University. Demos Many free demonstrations on Oracle Enterprise Manager Cloud Control 12c are available through the Oracle Learning Library. Here s a list of the demonstrations that are most relevant to this self-study. Other Resources The Oracle Learning Library offers many free demonstrations and tutorials. And, of course, the Oracle Enterprise Manager Cloud Control 12c documentation and online help embedded within the product are also valuable resources.