A Conditional Probability Model for Vertical Multithreaded Architectures

Transcription

1 A Conditional Probability Model for Vertical Multithreaded Architectures Robert M. Lane Abstract Vertical multithreaded architectures can be estimated using a model based on conditional probability. The advantages of such a model are simplicity, mimicry of reality, and accuracy. The model has been validated using industry standard benchmarks from several domains and vendors. The model generally establishes a performance high water mark. Large deviations in performance from the model s predictions suggest pathological behaviors that cannot be hidden by other threads (e.g., pipeline interlock issues, destructive cache interference, etc.). Introduction Vertical multithreading is a technique used by core architects to extract more performance from a core. Core architects understand performance in terms of instructions per cycle (IPC). The more instructions that can be executed every cycle, the happier they are. This implies that the core is doing more work and is more efficient. An IPC of 1.00 means that an instruction is executed every cycle, which is a perfectly efficient core. An IPC of 0.25 means that an instruction is executed every 4 cycles, which is not very efficient. The inverse of IPC is cycles per instruction (CPI). Core designers often ignore CPI; however, a breakdown of the CPI shows where workload time is spent. CPI comprises base instruction execution, pipeline interlock, and memory hierarchy accesses. This breakdown of time is informative and may be useful. Vertical multithreading exploits the fact that memory access speeds have not increased as fast as core speeds; consequently, memory hierarchy accesses have become a larger proportion of the CPI and the cause of low IPC and core inefficiency. Core architects recognized that the core sat idle for long periods while needed memory was fetched. But what to do with the idle cycles? One option was to allow heat to dissipate because as cores ran faster the heat generated could not be released fast enough. Another option was to throttle the speed, add strands, 1 and multiplex among 2007 by Robert M. Lane. All rights reserved. No part of this publication may be reproduced, stored in a retrieval system, or transmitted in any form or by any means for example, electronic, photocopy, recording without the prior written permission of Robert M. Lane. The only exception is brief quotations in printed reviews. 1. To avoid confusion between hardware and software threading, strands and vertical multithreading will be used throughout this paper to refer to hardware threading.

2 the runnable strands (cf. Figure 1, timings are illustrative and not realistic). Out of this latter option was born vertical multithreaded architectures. Figure 1 When vertical multithreaded architectures are deployed, application response times often degrade and throughput is not what is expected. System administrators and capacity planners are confronted with looming questions: How is performance of vertical multithreaded systems to be predicted? Why does response time and throughput degrade if the core is more efficient? At one company, workload experts were asked to predict the throughput and response time characteristics when migrating to vertically multithreaded systems. Their predictions were no better than chance even when they had access to system hardware experts. In some cases, the predictions were wildly off. This paper proposes a statistically based model to answer performance questions, to understand the interactions among strands, and to explain the difficulty of heterogeneous projections. It allows system administrators and capacity planners to understand if the system is performing close to expectation or wildly divergent. The Conditional Probability Model The conditional probability model is intuitively constructed based on the way that vertical threading works. The model is built in several steps to facilitate understanding and allow readers to apply the model to their situations. Derivation with Two Strands and Homogeneous Workload The conditional probability model arises from the observation that IPC is the probability of executing an instruction at a given time, which is a cycle in the units of measurement of a core. The IPC is independent of skew. At any instant of time that the core is checked, the IPC of a workload expresses the probability that an instruction will execute during that cycle. A workload has an IPC of The CPI is The core is a RISC processor so the base instruction execution is 1.00 cycles per instruction that is, an instruction requires one cycle to execute and the pipeline interlock and memory hierarchy overhead are 0.33 per instruction. If there are two strands that can execute the same workload with equal probability, then the initial model can be diagrammed

3 as a tree as shown in Figure 2. There is a 0.50 probability of trying to execute Strand A or Strand Figure 2 B. If the core chooses Strand A, there is a 0.75 probability of being able to execute an instruction and a 0.25 probability of not being able to execute an instruction. Figure 2 does not account for what happens on vertically multithreaded cores: when a strand cannot execute, the next available strand in the sequence is multiplexed to run. This is additional step is added to the tree diagram in Figure 3 Figure 3. The probabilities of executing an instruction for Strand A and Strand B are shown in the

4 Table 1. The total IPC for the core is Rarely will a core execute with an IPC of much better than due to pipeline interlock issues, branch misprediction, cache interference, etc. Table 1: Two Strands and Homogeneous Workload Strand A Strand B Strand A Strand B Start w/ Strand A 0.5* *0.25* Start w/ Strand B 0.5* *0.25* Total IPC These calculations give an indication about the throughput and response time for this homogeneous workload. Vertically multithreaded throughput is 25% higher than if the workload were executed on a single strand as determined by the improved IPC (calculated as 0.938/0.75). The improved throughput came a cost: poorer response time. The IPC for each strand is so the response time is 60% longer for each unit of work (calculated as 0.75/0.469). Two Strands with Heterogeneous Workload Generalized If there are two strands that can execute the two different workloads with equal probability, then the initial model can be diagrammed as a tree as shown in Figure 4. In fact, if the strands can be Figure 4

5 prioritized, the probabilities of initially selecting Strand A and Strand B can modified insofar as Prob StrandA + Prob StandB = The respective probabilities of executing an instruction during a cycle for Strand A or Strand B are calculated by walking the tree. Prob StrandA = 0.5 IPC A ( 1 IPC B ) IPC A Prob StrandB = 0.5 IPC B ( 1 IPC A ) IPC B If the workload running on Strand A has an IPC of 0.40 and the workload running on Strand B has an IPC of 0.20, effects can be determined using conditional probabilities. Furthermore, if the workload running on Strand A executes 750,000 instructions per unit of work and Strand B executes 500,000 instructions per unit of work, their performance can be predicted when run on vertically multithreaded hardware. The aggregate IPC is Table 2: Two Strands and Heterogeneous Workload Strand A Strand B Strand A Strand B Start w/ Strand A 0.5* *(1-.40)* Start w/ Strand B 0.5* *(1-0.20)* Total IPC The vertically multithreaded IPC for Strand A is 0.36 of which 0.16 is due to stealing cycles from Strand B when it could not execute an instruction. The vertically multithreaded IPC for Strand B is 0.16 of which 0.06 is due to stealing cycles from Strand A when it could not execute. The performance of the two workloads on vertically multithreaded and non-vertically multithreaded

6 Table 3: Two Strand Heterogeneous Workload Performance Comparison Single Core/Single Workload a Vertically Multithreaded b Concurrent Non-vertically Multithreaded c Strand A Strand B Strand A Strand B Strand A Strand B Response Time (cycles) % diff % +25% +100% +100% % diff 0.00% 0.00% % % % % a. Empirical single strand performance measured with other strands disabled. b. Derived values. Response Time = Instructions/tx * CPI. Throughput = GHz / Response Time. c. See note b above. cores can be predicted. With plenty of opportunity of overlap memory misses, a vertically multithreaded core gives both workloads opportunity to execute concurrently on a single core. Four Strands with Heterogeneous Workload The four strand conditional probability model is an extension of the two strand model. A tree diagram of the model is shown in Figure 5. The model easily translates to a spreadsheet to explore workload characteristics. Consider the behavior of a homogeneous workload with an IPC of 0.20, which is not unusual of workloads in the commercial enterprise space. When executed on a core with four strands, the aggregate IPC is Each strand has an IPC of Table 4: Four Strand Homogeneous Workload Strand A Strand B Strand C Strand D Total Base IPC Vertical MT Core

7 Figure 5

8 But what happens when the IPC of one of the strands, let us say the first, is increased to 0.50? The performance of the other strands becomes unpredictable. This is due to the idle time available in each strand and the interaction with the preceding idle time. No wonder workload experts had difficulty predicting performance characteristics. When workloads on vertical multithreaded systems have different IPCs, factors like processor affinity, process migration, and process binding can skew performance in disturbing ways. Table 5: Four Strand Heterogeneous Workload Strand A Strand B Strand C Strand D Total Base IPC Vertical MT Core Consider the combinatorics of four different IPCs running on a vertically threaded core: 0.20, 0.40, 0.60, and In Table 6 by reversing the positions of the 0.60 (red shading) and 0.80 Table 6: Four Strand Heterogeneous Workload (green shading) Base IPCs on the strands, the per strand IPC changes radically. This is usually deemed to be undesirable behavior because performance in inconsistent and non-intuitive. Benchmarks Strand A Strand B Strand C Strand D Base IPC Vertical MT Core Base IPC Vertical MT Core The conditional probability model has been tested in a number of industry standard benchmarks across a broad range of disciples and vertically multithreaded implementations. 1 Despite the model s simplicity, it has proven to be surprisingly robust. In most cases, the model is optimistic because it does not take into account secondary effects like destructive cache interference, TLB conflicts, pipeline issues, etc. These secondary effects tend to increase with the number of strands. The model has proven to be especially useful when comparing benchmark results with predicted performance. Differences of a 5-8% are acceptable given the simplicity of the model. Wide discrepancy have indicated severe problems that required root cause analysis using a CPI component cost breakdown using hardware performance counters normalized per instruction. Through performance counters and events, performance analysts and engineers can drill down to the problem area when vertical multithreading does not work as expected. 1. Benchmark results are proprietary. Engineers who have applied the model have remarked that it is surprisingly simple and accurate.

9 Conclusion The conditional probability model is intuitive when vertical multithreaded architectures are considered. The model is based in based in reality. It is not an artificial mathematical construct. Engineers can easily acquire the metrics for specific workloads to instrument the model. The model allows one to determine the upper bound of performance and whether measured performance is within acceptable bounds. If not, the model allows one the determine when performance is wide of expectations and troubleshoot the problem using hardware performance counters. The model can be extended to accommodate innovations. For example, vertical threaded implementation could over clock a strand when no other strand is available to run. The conditional probability model allows one to determine the probability that a strand will execute without another one being executable. Biographical Note Robert M. Lane is a performance analyst and software engineer with over twenty years of performance experience. Bob is currently working on a hardware performance book. As a Senior Staff Engineer at Sun Microsystems for over twelve years, he worked on strategic initiatives for new processor and system architectures and products that included databases, operating systems, storage systems, data collection and analysis frameworks, and system performance. He served as Sun s primary representative to the Transaction Processing Performance Council for eight years. Before that he worked on tactical and strategic performance at Pyramid Technology. Bob s contributions to the resolution of numerous customer and ISV escalations earned him numerous executive awards. Prior to joining Pyramid Technology, Bob contracted performance and software porting services throughout Silicon Valley.